What would this punnett square look like?

Do your punnet square like this
____AB     AB    ab     ab
AB
AB
ab
ab
Then fill in the crosses like a regular punnet square, there are just bigger.   the top left would be AABB and so on.

It sometimes helps to figure out how to pair the letters if you assign each letter a trait.  Say the letter A is for height (A is tall and a is short) and B is for hair color (B is dark and b is light).  A gamete has to have all the information necessary to make a new organism, with one half coming fom each parent.  So each gamete has to have the letters for each trait.  In your problem, the gametes would each have to have an "A" or a "B" whether the letter was upper or lower case (this will be determined by each parent's genotype).  It wouldn't do to have a parent make a gamete with genes for height only and the other parent do the same - then there wouldn't be any "instructions" for the hair color.   It's more efficient to have one full set of chromosomes from each parent.

So your first choice of AB AB and ab ab is correct.  This is easy because both alleles in each set are alike (both upper case for this parent).  If the parent was heterozygous for both traits (AaBb), this is where you use FOIL - pairing the First letter of each type, the Outside set, the Inside set, and the Last set.  So for AaBb the gametes will be AB, Ab, aB, and ab.