I believe it came from beer. Just like people who like to brew their own beer at home instead of drinking established brands, game hobbyists in the 90s started making their own "homebrew" versions of famous games like Mario and Zelda. Today the word homebrew is used for any unofficial app that's meant to run on a jailbroken or hacked console and not sanctioned by the console maker.