Using on and off switches and logic gates
It sounds to me that when a switch is open, you have a floating logic input. Logic chips, in general, do not like to have floating inputs and this can produce results like you see (gates half-way on).

Try adding pull-down resistors to your inputs, for example 4.7K between the input and ground. This guarantees that an input always sees a valid logic state.