Why do people often refer to Korean as the most difficult to learn?

(They're not Chinese characters, they're Japanese characters called kanji. Just needed to clarify haha)

As a native Japanese speaker I even have trouble reading adult books as there are so many kanji that I don't know even though I have graduated Japanese elementary, middle, and high school. I often have to label kanjis that I forgot or simply don't know with hiragana on the side to remind myself how to pronounce them(but not all the time). I agree with you that Japanese is the most difficult language to learn from a reading perspective. Korean is much much easier to read. (Although if you exclude kanji, the reading might be equal on the difficulty scale because Japanese simple kana characters are all pretty much unique in shape and they each represent 1 syllable only, so you automatically know just by glancing how many syllables there are and what they are)

I think that some people get confused because they hear that Japanese is the easiest East Asian language to pronounce, which is true in my opinion. I guess that somehow transforms in their minds into the easiest East Asian language overall (which isn't true imo). I think all East Asian languages are difficult and easy in their own way; to compare Japanese and Korean, Japanese is easy to pronounce but hard to read, while Korean is easy to read and somewhat difficult to pronounce(?). I don't speak Korean so I am not sure, but when I listen to KPop (KPop is great), like song lyric videos, even though I can basically understand how to read rom it's way too fast and the combination of sounds is hard to say fluidly for me.

I have no idea, because I think the same way! Except for me I think Chinese is the most difficult out of the three, but I'm learning Korean and it hasn't been that hard.

It is not objective... Everyone has different appreciation