What settings should I use in my.cnf
InnoDB
You need to run this query:
SELECT
    CONCAT(IB_BB/POWER(1024,1),'K') IB_KB,
    CONCAT(IB_BB/POWER(1024,2),'M') IB_MB,
    CONCAT(IB_BB/POWER(1024,3),'G') IB_GB
FROM (SELECT SUM(data_length+index_length) IB_BB
FROM information_schema.tables WHERE engine='InnoDB') A;

This will tell you what to set for innodb_buffer_pool_size or 75% of the DB Server's total memory, whichever is smaller.
See also

Jan 13, 2012 : What to set innodb_buffer_pool and why..?

MyISAM
You need to run this query:
SELECT
    CONCAT(KeyBuffer_BB/POWER(1024,1),'K') KeyBuffer_KB,
    CONCAT(KeyBuffer_BB/POWER(1024,2),'M') KeyBuffer_MB,
    CONCAT(KeyBuffer_BB/POWER(1024,3),'G') KeyBuffer_GB
FROM (SELECT SUM(index_length) KeyBuffer_BB FROM information_schema.tables
WHERE engine='MyISAM' AND table_schema NOT IN
('information_schema','performance_schema','mysql')) A;

This will tell you what to set for key_buffer_size.
CAVEAT #1
Use common sense about using both buffers in relation to the total installed RAM (See my answer to What are the main differences between InnoDB and MyISAM?)
CAVEAT #2
Once you set these values and start mysql, please download mysqltuner.pl and run it. It will forecast how much RAM is need for all buffers and for connections (it terms of buffers joins, sort, reads)
CAVEAT #3
As for InnoDB, there are other settings you need to set for getting InnoDB to use
Multiple CPUs/cores:

May 26, 2011 : About single threaded versus multithreaded databases performance
Sep 12, 2011 : Possible to make MySQL use more than one core?
Sep 20, 2011 : Multi cores and MySQL Performance

InnoDB Tablespaces for Individual Tables

Dec 09, 2011 : What is the best way to reduce the size of ibdata in mysql?
Feb 22, 2012 : Ibdata usage and Recommendations?
Apr 01, 2012 : Is innodb_file_per_table advisable?
Apr 19, 2012 : MySql - Clean ibdata1
Oct 22, 2012 : How large should be mysql innodb_buffer_pool_size?

and so much more