Subluxation (image on the left) means that the kneecap almost dislocates - i.e. slips up the side of the underlying groove of the femur bone - but does not quite slip over it.
In dislocation (image on the right) the kneecap comes right out of the underlying trochlear groove and with the first episode there is usually tearing of the restraining structures on the inner side ('medial retinaculum').
The term 'subluxation' is actually used a little loosely. A surgeon may indicate 'subluxation' on an x-ray, when indicating that part of the patella habitually rides outside the underlying rim of the trochlear groove -
But a surgeon may ask if you 'sublux', in this case meaning that you periodically feel the patella suddenly - and uncomfortably - jump partly over this rim. This is a discomfort, rather than something you can see.
A surgeon may also, when examining the knee, try to push the kneecap sideways out of the groove in an attempt to physically 'sublux' the patella. This is to measure the degree of restraint by the capsule and the retinaculum (the thicker bits on the side). If the patient anxiously restrains the examiner from doing this, it is termed a 'positive apprehension test', and suggests that the patient has previously experienced pain from instability.