After OSTimer() is called in an interrupt, does control return to the scheduler, or does execution resume where it left off?
It resumes where you left off. OSTimer() is very short -- perhaps 20 cycles or so. So, in your original scenario, when OSTimer() is called and the LED update task becomes eligible again, it will not run until the "slow" multiply task is finished and yields back to the scheduler via OS_Yield().
Obviously, an interrupt-driven LED update will have the least jitter, and may also be the smallest solution, ROM-wise. Be sure to code it entirely in the module that contains the interrupt function so that PICC saves only the minimum register set.
If you can tolerate some jitter (I'm assuming you'll be running at 20MHz / 200ns/instruction, since you have a relatively fast timer tick of 2ms), then you could simply add a couple of OS_Yield()s to the multiply task. 2ms @ 20MHz is 10,000 instructions. Even for a slow multiply task, that's a long time. By spacing a couple of OS_Yield()s evenly (instruction-count-wise) in your slow multiply task, then you can do the LED updates from a task and end up with fairly minor jitter.
If you have something like a = b * c * d, you could do:
a = b * c;
a *= d;
The idea being that since OS_Yield() is only callable at the C level (not at the assembly / machine instruction level), by splitting the operation into a couple of steps, you can interject OS_Yield()s between PICC library calls (to the multiply routine). Your ROM will only grow (roughly) by the size of an extra OS_Yield() ... not really a big deal. The actual number of cycles spent multiplying won't change much either, though you will have extra trips through the scheduler ... but it's a slow / low priority task anyway ...
All interrupts are of course of higher priority than any task. I'm not sure why you need to update an LED at 500Hz. Assuming that it's a human-interface issue, I would assign the LED update task a moderate priority, so that the system can still have even more important tasks and interrupts at a higher priority than the LED updates.
Just some thoughts ...