Calculating frequencies m/c full points for answer + explanation please?

I'm afraid the answer is none of the above, because it is impossible to have a population in Hardy-Weinberg equilibrium in which the hybrids number less than either the homozygous recessive or homozygous dominant.  Therefore, there was probably a sampling error (or your teacher was making up numbers).  

The background:  You have 2 of each kind of chromosome, one inherited from your mother, one from your father.  Both the chromosomes have the same genes.  For example, if the gene is for eye color, there is one of that kind of gene on each of the chromosomes (regardless of what information is actually there).  Let's say there are two alleles, one for brown eyes, which is dominant B, on one for blue eyes, which is recessive b.  (let's keep it simple and say those are the only two possibilities, even though it's not like that in real life - there are actually multiple genes that influence eye color, which is why you could actually have gray or hazel colored eyes).  

Now, let's say there are only two possibilities (brown or blue).   We will call the frequency of the brown allele in the population p and the frequency of the blue allele in the population q.  Obviously, because there are only two possibilities, p + q must equal one (or 100%).  If p is .1, then q must be .9, and so forth.  

However, you get two chromosomes, so you have two alleles.  Because there are only 2 alleles, you could only be BB, Bb, or bb.  To express this mathematically, (p + q) = 1, therefore (p + q)^2 =
1^2, which in expanded form is  p^2 + 2 pq + q^2 = 1.  p squared (p^2) is the probability that an individual will have two B alleles ( be BB), q squared (q^2) is the probability that an individual will have two b alleles (bb) and 2 pq is the probability that an individual will be a hybrid (Bb).  

Let's look at an example.  If the frequency of the brown eyed allele and the blue eyed allele are the same, they would be 0.5.  (Because 0.5 + 0.5 = 1).  Expanded, you would have a population of about 0.25 BB, 0.5 Bb and 0.25 bb.  

If given population numbers for each genotype, if you have to figure out the frequency of either gene, you would take the square root of the population numbers of individuals with either of the homozygous traits.

Let's look at your teacher's example.  Is there any combination of values for p and q in which you could get a value of 2pq that is less than p^2 and less than q^2?  Remember that p + q must equal 1.  If p is greater than q, then p^2 must be larger than 2 pq.  If q is greater than p, then q^2 must also be larger than 2 pq.  And (as in our example above), if p = q, then 2 pq MUST be larger than either homozygous state.  

There's a good you tube video on the Hardy Weinberg equilibrium that explains all the assumptions necessary.  Look at: