I think the difference in American English is that jam has fruit pieces in but jelly doesn't. In British English they're both called jam, and "jelly" means something different, which in the US is called Jello.
This is jam, because it contains pieces of fruit
Americans would call this jelly, but people from the UK call it jam
In the UK, we call this jelly. Americans call it Jello.