How do you drive a 1 watt LED?
Use an LED DC-DC driver IC. They're cheap, and you won't be shedding so much power in the the regulator. This one would work well: https://www.diodes.com/products/power-management/led-drivers/low-voltage-dc-dc-led-drivers/part/PAM2804

Another benefit of using this kind of regulator is that they set current in the LED, not voltage. Thus they eliminate the need for a load resistor.

If you use a linear regulator like the LM317, the total power loss (regulator and the load resistor) will be (5.2-3.2V)*0.35A = 0.7W, not a small amount of loss (41%, or 59% efficiency.) This is true regardless of how you use the regulator (constant-current or constant-voltage.)

One more thing: The LM317 has a high dropout voltage, about 1.6V for that current. This leaves just 0.4V of margin for it to operate the LED.