Sounds like DNS problems.
Is the modem a true modem or is it a router too, a lot of cable providers are combining a router with firewall on the 'modem'.
When you connect direct to the modem with 'USB or ethernet' how are you setting the network settings on the PC? Do you have to manually set the ip, dfgw, subnet and dns? If you do then make sure that's in the linksys router also.
If you're setting up using DHCP, that is the PC is set to automatically set its entries for the conection from the 'modem' you have a combined modem router.
For the Linksys router, they're usually pretty good. Sounds like a dns or gateway problem as said before so check you've got the right ip, subnet, default gateway and DNS entries in the router which will need to be manually set in all circumstances.
If you've got a combined modem/router with dhcp then the linksys will see a private network range on its WAN side, you've got to set another range on the linksys private side.