What structures of the integumentary system or skin is involved in regulating temperature?

Sweat glands and pores. Read the first paragraph at Discovery Heath (link below).

The integumentary system includes the skin and the related structures that cover and protect the body. The human integumentary system is composed of the skin, and includes glands, hair, and nails. The largest organ in the body, the skin protects the body, prevents water loss, regulates body temperature, and senses the external environment. The thing that basically helps heat you up is goosebumps.

Goose bumps are created when tiny muscles at the base of each hair, known as arrectores pilorum, contract and pull the hair erect. The reflex is started by the sympathetic nervous system, which is in general responsible for many fight-or-flight responses.

As a response to cold: in animals covered with fur or hair, the erect hairs trap air to create a layer of insulation. Goose bumps can also be a response to anger or fear: the erect hairs make the animal appear larger, in order to intimidate enemies. This can be observed in the intimidation displays of chimpanzees,[1] in stressed mice[2] and rats, and in frightened cats. In humans, it can even extend to piloerection as a reaction to hearing nails scratch on a chalkboard, listening to awe-inspiring music,[3] feeling or remembering strong and positive emotions (e.g., after winning a sports event),[4] or to an intellectual or spiritual epiphany.
Piloerection as a response to cold or emotion is vestigial in humans; as humans retain only very little body hair, the reflex (in humans) now provides no known benefit.


However the medulla oblongata is primarily responsible for regulating body temperature and is the back part of your brain but just giving you an FYI, this is not part of the integumentary system however plays a key role though.

Your skin has sensory receptors that are sensitive to temperature changes in the outside environment. These kinds of receptors are called thermoreceptors and these are found either in the dermis or epidermis.

When the temperature drops, these thermoreceptors send a message to the temperature regulating center of the brain, which is the hypothalamus. The hypothalamus then sends out signals to the blood vessels in the skin, as well as to the arrector pili muscles and this causes the skin to contract and . The arrector pili muscles are attached to the hair follicles so when they contract, they pull on the hair and you get goosebumps. When you get goosebumps, the heat becomes trapped between the erect hairs and this helps your body retain heat. Your body stops producing sweat (because your hypothalamus ordered your sweat glands (also located on your skin) to do so) and you shiver.

When the temperature rises, these thermoreceptors again send a message to the hypothalamus. The hypothalamus orders your sweat glands to start producing sweat. The sweat oozes out of the pores of your skin and when it evaporates, your body cools off. The message is sent to the blood vessels on your skin and arrector pili muscles too, but the reaction is different this time around. The blood vessels dilate, causing increased blood flow to the surface capillaries of the skin. This allows the blood to 'cool off' through the process of convection and conduction. The arrector pili muscles relax and this causes the hair to lie flat on your skin, allowing more air to skim over its surface.

So all in all, the structures of the skin involved in thermoregulation include:
Thermoreceptors
Arrector pili muscles
Sweat glands (apocrine and eccrine)
Blood vessels located on your skin
Hair follicles