Scribble Art. I did this project with a group of 3 and 4 year- olds and several of the lanterns turned out very pretty.
We covered the jars with tissue paper and mod podge. I cut letters out of black construction paper for the kids to glue on to make them more Christmas-y. (I had intentions of making a "love" lantern and a "peace" lantern as well, but haven't quite gotten around to those yet)
Mod Podge is non-flamable; so you could fill the lanterns with regular candles, but for kids' projects I really love the battery- operated tealight candles you can buy now. They are safe and pretty convincing.
Any other fun ideas for reusing baby food jars? (especially Christmas related would be fun!)