Cloudera CDH3 installation failure, how to get around this?
Question: I am attempting to install CHD3 onto a 3 node cluster. I launch the
installations via the Cloudera Manager. All three installations fail.

I see this error after the Cloudera installation fails in /var/log/cloudera-
scm-agent/cloudera-scm-agent.out:

    
    
      File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 19, in <module>
        import psutil
      File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/psutil-0.3.0-py2.6-linux-x86_64.egg/psutil/__init__.py", line 84, in <module>
        TOTAL_PHYMEM = _psplatform.phymem_usage()[0]
      File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/psutil-0.3.0-py2.6-linux-x86_64.egg/psutil/_pslinux.py", line 122, in phymem_usage
        percent = usage_percent(total - (free + buffers + cached), total,
    TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'
    

Apparently the Python interpreter running on start up sees "free", "buffers",
or "cached" as having a NoneType and this error causes the entire installation
to roll back.

Can anyone advise as to why this occurs and/or a way around the problem?

Thanks in advance.


Answer: The problem is here, in phymem_usage() in _pslinux.py:

    
    
    def phymem_usage():
        # total, used and free values are matched against free cmdline utility
        # the percentage matches top/htop and gnome-system-monitor
        f = open('/proc/meminfo', 'r')
        try:
            total = free = buffers = cached = None
            for line in f:
                if line.startswith('MemTotal:'):
                    total = int(line.split()[1]) * 1024
                elif line.startswith('MemFree:'):
                    free = int(line.split()[1]) * 1024
                elif line.startswith('Buffers:'):
                    buffers = int(line.split()[1]) * 1024
                elif line.startswith('Cached:'):
                    cached = int(line.split()[1]) * 1024
                    break
            used = total - free
            percent = usage_percent(total - (free + buffers + cached), total,
                                    _round=1)
            return ntuple_sysmeminfo(total, used, free, percent)
        finally:
            f.close()
    

Note that it is examining /proc/meminfo and is converting fields to integers
without checking if those fields exist. On some systems, including some
virtualization technologies, Buffers or Cache may be missing. (The LSB spec
states that most of these fields are optional.)

A quick fix would be to change /proc/meminfo to /tmp/meminfo, "cat
/proc/meminfo >/tmp/meminfo", and add a line like:

Buffers: 0 kB

