Understanding voltage and current of a phototransistor w/ collector resistor
Read the datasheet.  It should tell you what the expected phototransistor current will be.  That is apparently not enough to cause 5 V across a 10 k&Omega; resistor.

Do the math.  At full illumination, the A/D reads 900 out of a 0-1023 full scale range, which is 5 V.  That means the output is at 4.4 V, which means that there is 600 mV across the 10 k&Omega; resistor, which means that the current is 60 &micro;A.  That is apparently the current your phototransistor can support with the illumination you are supplying.

There are several things you can do about this:

Nothing.  As things are now, you can read the illumination to one part in 123, or basically 7 bits.  Is that perhaps already good enough?  What are you really trying to measure.  For many applications, 7 bits plenty.

Use a larger resistor.  In this case, the pullup resistor is being used as a current to voltage converter.  You can think of the resistance as being the gain of the conversion.  A 20 k&Omega; resistor would give you twice the voltage, for example.

The downside to this is that the impedance of the voltage signal goes up.  Check the A/D datasheet to see how high a source impedance it is specified to work with.  For some A/Ds, higher source impedance can be tolerated if the acquisition time is lengthened.  Again, consult the A/D datasheet to see what your A/D needs.

Amplify.  You have a 600 mV signal that you want to be closer to a 5 V signal.  That's exactly what amplifiers do.

In that case, I'd probably flip the phototransistor and resistor so that the signal starts at 0 V with no light and goes up with more light.  A gain of 8 would just about do it.

If the value at low light levels is important, then make sure to use a opamp that can handle signals down to ground on both the input and output.  Or, use a small negative supply from a charge pump or something so that 0 V is well within the opamps linear region.

Use a higher resolution A/D.  Many microcontrollers have 12 bit A/Ds built in nowadays.  Changing only the 10 bit A/D to a 12 bit A/D gives you 4x the resolution, or about 9 bits in this case.  Is that enough?