Stay hydrated! Vending machines and conbini are everywhere, so there's plenty of drink options if you need something refrigerated or don't want to carry a water bottle. Drinks like Pocari, CC, and Green Dakara are more hydrating than teas or sodas.
Coolish ice cream (in the freezer at any conbini) is also very refreshing, and because of how it's packaged, you can use it as a cold-pack in a pinch.
Uniqlo stocks tons of heat-reducing undershirts and things that I'm told work very well, and I've gotten "cooling" leggings from Tutu Anna (sock store) that literally make your legs freezing cold. Also look for umbrellas, sleeves, etc. labelled "UV cut" to protect your skin.
Bic Camera or Yodobashi are a good bet for finding a hand-held battery-operated fan. Stores like Daiso, Donki, Loft, or Hands may also stock them (or wearable neck fans), but IME chances and selection were better at a dedicated electronics store.
Good advice so far. In addition to the heat and humidity, watch out for the sun, which is more intense than you might expect. Stay in the shade when possible. You can find cheap wide-brimmed hats at a hardware store or garden center that will protect your head and neck from the sun. They’re unfashionable, but as a tourist you’ll probably be wearing shorts anyway, so who cares. At the end of your trip you can toss them or give them away.
Be alert for signs of heat exhaustion. If you feel disoriented or confused, stop whatever you are doing, get to a cool shady spot, and hydrate. I know Japan can be disorienting and confusing even in the winter, but it pays to be proactive.
