How do you keep lights from affecting them?
If it's truly infrared and not sensitive to visible light, no lights should have any effect on it. I suspect that you have a plain photocell and not an Infrared detector.
Can you provide a photo or part number or information about the circuit it is a part of?