Ice can come in many colors, including clear. You'll need to determine if your ice has color and what that color palette is, then you can start making ice.
There are several ways I know of to make ice.
Clear caulk has already been suggested, and that works well.
Woodland Scenics makes a product called water effects that can be spread out kinda stringy like over wax paper (So you can easily peel it off after it dries) and allowed to dry to make icicles. Most will not be the correct shape, but you get better with practice. The trick is to do it kind of like dry brushing and keep the brush moving in only one direction. The brush gets ruined, so don't use a good one. I think it's tintable with paint.
I found the the woodland scenics stuff was kind of expensive for the waste you get, so I did the same thing with clear 5 minute epoxy. It's cheaper, more readily available, and it dries enough to add it to the layout in about 5 minutes. I've just started experimenting with it, but it looks good so far. I haven't tried tinting it with paint, but I'm sure you can.