How can I add a power-off delay to this circuit
While you didn't provide a schematic, tsk, tsk ;) it's pretty obvious from your text what you were trying to accomplish with that relay, switch, capacitor and resistor.

Here's something that'll do what you want:



THE CIRCUIT

When S1 is made, Ct charges to nearly Vcc quickly, through R1, and when its voltage rises to the point where the voltage on the inverting input (-) of U1 is more positive than the voltage on U1-, U1's output will go low, turning Q1 ON.

That'll energize K1 and cause its common (C) terminal to disconnect from the Normally Closed (NC) contact and connect to the Normally Open (NO) contact for as long as S1 is made.

When S1 is released its contacts will open and Ct will discharge through Rt until the voltage on U1- goes more negative than the voltage on U1+, at which point U1's output will go high, turning Q1 OFF and de-energizing K1, which will disconnect C from NO and connect it back to NC.

THE TIMING 

If a charged capacitor is discharged through a resistor in parallel with the cap, the voltage across the capacitor will decay  to \$\approx\$ 37% of the charged voltage in one time constant, that time being equal to: 

$$t=RC$$

where t is the time in seconds, R is the resistance in ohms, and C is the capacitance in farads.

The choice of either R or C is arbitrary as long as the RC product equals the desired timeout, and I tend to prefer smaller capacitances as long as the resistances don't get outrageously large so, for this application, I chose 1 microfarad for Ct and 5.1 megohms for Rt as a comfortable compromise. The capacitor should be something other than electrolytic in order to keep the cap's leakage current from competing with the discharge current through Rt. 

Next, since the switching point will be about 37% of the supply voltage (which Ct was charged to [the supply voltage] before it started being discharged), U1+ needs to be set at about 2.2 volts.

That voltage will be provided by the the voltage divider R2R3, and since it'll be drawing current from the supply all the time, I chose 1 megohm for R1 to keep the drain small- around 6 microamps. 

to solve for R3 we can write:

$$ R3 = \frac{V_{REF}\ R2}{Vcc - V_{REF}} = \frac{2.2V\times 1\cdot10^{6}\Omega}{6V- 2.2V} \approx 579 \ k\Omega $$ 

560k and 620k are the closest E24 values and either would probably be OK for imprecise RC timing like this.

Just for grins, here's the LTspice schematic and plot; the green trace is S1 being pushed and released, and the red trace is the current through the relay coil.

The LTspice circuit list follows just in case you want to play with the circuit.

  

Version 4
SHEET 1 880 680
WIRE -960 -256 -992 -256
WIRE -768 -256 -960 -256
WIRE -544 -256 -768 -256
WIRE -416 -256 -544 -256
WIRE -144 -256 -416 -256
WIRE -768 -240 -768 -256
WIRE -544 -224 -544 -256
WIRE -768 -144 -768 -160
WIRE -816 -128 -880 -128
WIRE -144 -80 -144 -256
WIRE -416 -64 -416 -256
WIRE -768 -48 -768 -64
WIRE -640 -48 -768 -48
WIRE -448 -48 -640 -48
WIRE -320 -32 -384 -32
WIRE -208 -32 -240 -32
WIRE -544 -16 -544 -144
WIRE -448 -16 -544 -16
WIRE -768 32 -768 -48
WIRE -640 48 -640 -48
WIRE -544 48 -544 -16
WIRE -144 48 -144 16
WIRE -64 48 -144 48
WIRE -992 64 -992 -256
WIRE -880 64 -880 -128
WIRE -144 80 -144 48
WIRE -64 80 -64 48
WIRE -992 192 -992 144
WIRE -880 192 -880 144
WIRE -880 192 -992 192
WIRE -816 192 -816 -80
WIRE -816 192 -880 192
WIRE -768 192 -768 112
WIRE -768 192 -816 192
WIRE -640 192 -640 112
WIRE -640 192 -768 192
WIRE -544 192 -544 128
WIRE -544 192 -640 192
WIRE -416 192 -416 0
WIRE -416 192 -544 192
WIRE -144 192 -144 144
WIRE -144 192 -416 192
WIRE -64 192 -64 160
WIRE -64 192 -144 192
WIRE -992 288 -992 192
FLAG -992 288 0
FLAG -960 -256 6V
SYMBOL ind -80 64 R0
SYMATTR InstName L1
SYMATTR Value 1
SYMATTR SpiceLine Rser=72
SYMBOL diode -128 144 R180
WINDOW 0 24 64 Left 2
WINDOW 3 24 0 Left 2
SYMATTR InstName D1
SYMATTR Value 1N4148
SYMBOL res -224 -48 R90
WINDOW 0 0 56 VBottom 2
WINDOW 3 32 56 VTop 2
SYMATTR InstName R3
SYMATTR Value 1000
SYMBOL res -560 -240 R0
SYMATTR InstName R1
SYMATTR Value 1meg
SYMBOL res -560 32 R0
SYMATTR InstName R2
SYMATTR Value 600k
SYMBOL res -784 16 R0
SYMATTR InstName Rt
SYMATTR Value 5.1meg
SYMBOL sw -768 -48 M180
SYMATTR InstName S1
SYMBOL voltage -880 48 R0
WINDOW 3 24 96 Invisible 2
WINDOW 123 0 0 Left 2
WINDOW 39 0 0 Left 2
SYMATTR Value PULSE(0 1 1 10m 10m 500m)
SYMATTR InstName V1
SYMBOL Misc\\battery -992 48 R0
WINDOW 123 0 0 Left 2
WINDOW 39 0 0 Left 2
SYMATTR InstName Vcc
SYMATTR Value 6
SYMBOL res -784 -256 R0
SYMATTR InstName R4
SYMATTR Value 1000
SYMBOL pnp -208 16 M180
SYMATTR InstName Q1
SYMATTR Value 2N3906
SYMBOL Comparators\\LT1716 -416 -32 R0
SYMATTR InstName U2
SYMBOL cap -656 48 R0
SYMATTR InstName C1
SYMATTR Value 1µ
TEXT -976 224 Left 2 !.model SW SW(Ron=.01 Roff=1G Vt=0.5Vh=0)
TEXT -976 256 Left 2 !.tran 10 startup uic