Describing a physical system in idealized terms involves making literally false claims about the system. Given this, it is puzzling that justified beliefs about physical systems can be formed by starting with idealized descriptions and then performing mathematical calculations. I argue that this puzzling aspect of idealizations cannot be easily removed by introducing talk of approximations. I go on to develop an account of how this curious feature of idealizations is to be understood. My account requires us to reassess what precisely we take the laws of physics to be saying, and also has consequences concerning the kind of evidence we can have for thinking that mathematics is the 'language of nature'. Finally, some critical comparisons are made with the so-called model-based account of scientific laws developed by Cartwright and Giere.