There are three sources of noise, which manifests itself as a graininess in your images.
First, all light has noise, much like static in a radio signal, but this isn’t something you can see with the naked eye. This is referred to as “shot noise” and it varies in accordance with the magnitude of the light. If you have lots of light, the light to noise ratio is pretty much insignificant, but when you have less light, there is proportionally more noise. So shooting at a higher ISO cuts the amount of light that’s recorded, increasing the ratio of noise to light. Of all three sources of noise, shot noise is the most significant.
The second source of noise is “read out” noise, i.e. inaccuracies in reading the voltage at each pixel. This doesn’t vary with ISO, but any noise introduced at this stage is magnified proportionally as you increase the ISO. This source of noise can be minimized by slowing the read rate from the sensor, but this isn’t a practical solution when you need a camera with a reasonably high frame rate.
The third source is “dark-current noise.” When a photon hits the sensor it creates an electron, which is then converted to a voltage, but heat can create additional electrons thereby adding yet more noise to the system. Scientific cameras solve this problem by cooling the sensor, sometimes to as much as -100C, but this clearly isn’t a practical solution for a consumer camera.
In short, then, noise is inevitable, but there are two additional factors that determine the extent to which it will be problematic: the size and the age of your sensor. Smaller sensors, particularly those in compact digital cameras, tend to produce more noise because the photosites are considerably smaller: as such, they are less accurate. Older cameras also produce more noise than newer ones. For example, the ISO range of the Canon 20D, which was released in 2004, is 100 to 3200, but ISO 3200 is almost unusable due to the high amounts of noise it generates. The Nikon D4s on the other hand—released in 2014—has an effective ISO range of 200–25,600 (and can be boosted to 409,600!). This camera will produce much cleaner images at higher ISOs than the 20D as sensor technology has advanced considerably in recent years. In each case, though, irrespective of the type or age of your camera, increasing the ISO will introduce some additional noise, and the higher the value you use, the more noise it will introduce.
In summary, when changing the aperture or shutter speed to obtain an optimum exposure is either impossible or undesirable, changing the ISO can be a viable alternative, but it will introduce at least some degree of noise.