Need help factoring this polynomial completely?

The problem is you didn't completely factor at the beginning. You should have factored an x^3y, not xy.

Heres the work:

x^3y( x^2y^2 - 9xy + 20)

You would then factor ( x^2y^2 - 9xy + 20), which is

(xy - 4)(xy -5)

so the final answer is

x^3y(xy - 4)(xy -5)

yes this problem can be a bit confusing. here is how i solved it:

x^5y^3 - 9x^4y^2 + 20X^3y

factor out GCF: x^3y(x^2y^2 - 9xy + 20)

now, this looks like it should be hard to solve, but figure that xy=a

factor the inside: (a^2 - 9a + 20) = (a-4)(a-5)

replace a with xy: x^3y ( xy - 4 ) ( xy - 5 )

From the original polynomial, you need to factor out as much as possible (x^3y):

x^3y(x^2y^2 - 9xy + 20)

From there I factored out the latter part using the foil method:

x^3y(xy-5)(xy-4)

Hope this helps!