I’d assume we’d have to get into the nitty of how things work.

If it’s a field that bends light around a perimeter then the light from the torch would respond much like light reflected off shiny armor, and get displaced. That said, if light isn’t absorbed by the inside but reflected back onto whatever’s inside it, it’ll get toasty with normal body heat, let alone a burning fire. An invisibility cloak might double as a thermal cloak. Space blankets (that is, aluminized plastic) work based on this principle.

So a cloak such as Harry Potter’s Invisibility Cloak might not just be configured to bend light but also regulate the environment under the cloak, discretely venting heat / collecting heat as necessary to keep the wearer comfortable. Otherwise it might be like the option considered in Sneakers (1992 film staring Robert Redford) when the sneak team was confronted with the challenge of getting past high-end motion detectors / heat cameras:

We can wrap you in a full body suit of neoprene, heat-resistant rubber…
…or raise the temperature in Cosmo’s office to 98.6 degrees…
…which is probably what we’ll do. The neoprene would suffocate you…

Then there’s mental invisibility, which convinces the observer that I’m not here! e.g. an SEP field or Jedi mind trick. This would block the beholder that the light source doesn’t exist but would still allow them to see by it, which can create cognitive dissonance that would eventually alert the observer that something is amiss. Depending on the details (and whether it’s relevant to the story) the beholder might be able to realize what is going on, even as their brain is blocked from comprehending it fully.

I’m reminded of a play-through of mine in The Sims in which I was managing the house of a lesbian couple. One of them wanted kids, and as an experiment I had her woohoo (cheat, have sex) with a man and get pregnant to see if her partner would notice something was a bit off. She never suspected a thing.

There’s an example similar to this in Wild Cards: Joker’s Wild (1987 gritty super-hero genre, edited by George R. R. Martin) in which the Astronomer whammies Fortunato so he can’t perceive the Astronomer in any way. In a late chapter, it starts driving Fortunato crazy since the Astronomer is among the service team at the restaurant Aces High, and he notices people near him are being served when all the visible servers are busy with other tasks.

🤓