Cool C Programming - <AUTHOR> http://sathyamvellal.in/blog/cool-c-programming/?d=5 <comment><USER2> I am a bit rusty, but isn't it always better to individually parenthesize macro variables in the expression?

      #define CHECK(X, Y) \
      do { printf("Performing Assertion\n"); assert((X - Y) != 0); \
      printf("Assertion passed\n"); } while (0)

Here, assert((X - Y) != 0) should be assert(((X) - (Y)) != 0)

It's OK when you are doing subtraction, but if you were doing something like division or multiplication, you could screw with the order of operations if you don't do this.

<comment><AUTHOR> Yes, macros are to be fully enclosed but just for a better readability sake I've avoided them here. As I said in my comment above, I mainly intended to convey the idea. I did explain every aspect during the talk though. I'll make a better effort from next time! Thanks

<comment><USER2> This seems like a little detail that there's no harm in including, especially since you focus so much on macros, and this is kind of macro 101.

<comment><AUTHOR> Yes, I get your point :) The post doesn't focus on the preprocessor actually, its just that there's more to explain in it which people usually neglect. So I've somehow written more on the preprocessor which looks like I've focused on it.

</comment></comment></comment></comment><comment><USER3> I wish this article talked about why people choose to do and not do these things.

So much of this stuff is the kind of thing people new to C should not be doing. Minimizing use of preprocessor directives leads to more maintainable C. Use them where they are needed and not anywhere else.

Also, for the swapping they show:

      a = a ^ b;
      b = a ^ b;
      a = a ^ b;

They fail to point out it's (platform dependent) performance implications. The above code is faster then the below code on PowerPC/POWER for example, but slower on x86/amd64.

      int tmp = a; //or: register int tmp = a;
      a = b;
      b = tmp;

<comment><USER4> Too many older developers are carrying around biases today that used to be true (like the performance issue you pointed out).

<comment><USER5> I'm still trying to shake off their influence. :P

</comment></comment></comment><comment><USER6> Hah, that's a fun one:

      double array[SIZE][SIZE] = {
          #include "float_values.txt"
      }

Been there done that.

<comment><USER7> Meh, there is already an idiom for that. It's

#include "data.h"

Where data.h has "extern double array[SIZE][SIZE];" and data.c has the data. I don't see what's clever or useful about this.

<comment><USER6> Here, "float_values.txt" could just be a pile of comma-separated numbers, suitable for editing by non-programmers.

It's a cute hack.

<comment><USER7> I've worked in a lot of situations with C/C++ code and non-programmers, and that simply wouldn't work. Non-programmers will break your build with such a mechanism. If they are capable of editing comma-separated lists of numbers in a text editor without breaking it, they are capable of editing a C file with a float[][] declaration at the top.

It would cause a problem after about 2 days, and then you would end up replacing this in 5 minutes with a 5 line Python script that does a minimal amount of error checking (or a higher level tool to generate the data, e.g. a game logic editor).

EDIT: A cute hack is one that lets you take a shortcut and saves you time. This is one that saves a tiny amount of typing (or writing 5 lines of Python), in exchange for wasting time with broken builds.

<comment><USER8> Taking that pattern to the next level is Mozilla Spidermonkey, which defines a macro and then includes a static data file which calls the macro with its data contents. The same table is included multiple times with different macro definitions to output different code. Pre-processor code generation :) Example:

[http://hg.mozilla.org/mozilla- central/file/48dbd532a004/js/s...](http://hg.mozilla.org/mozilla- central/file/48dbd532a004/js/src/jsopcode.cpp#l60)

<comment><USER9> That's pretty normal, if used in moderation.

For example, it is a way to define overloads on functions on vectors with floats and double coordinates without having to copy-paste thousands of lines (using sed and make is another)

[http://www.drdobbs.com/the-new-c-x- macros/184401387](http://www.drdobbs.com/the-new-c-x-macros/184401387) gives nice examples.

</comment></comment></comment></comment></comment></comment><comment><USER4> This is a pretty basic article, but might be useful for some. One point I take issue with: a pointer in C is not simply a variable containing an address. It also knows (at compile time) how big the object pointed to is and so increments and decrements mean different things depending on the pointer type. In addition, void* is special because it does NOT know it's size.

<comment><USER10> a pointer to anything, no matter what type it is, is ALWAYS the same size as a variable or embeded in a struct, (8 bytes in 64 arch and 4 bytes in 32 bit) cause it only needs to keep the address in memory to the thing its pointing to.. thats where the 4GB of memory limit for 32 bit architecture came from..cause you could not point to any memory address that would not fit into a 4 byte int

i think what you are saying is the thing the pointer is pointing to, dont have a known size if its a pointer of void.. since its the only untyped type of pointer.. and the programmer would need to check the size of the thing being pointed in memory at runtime.. (so sizeof wouldnt do it)

Also, if you incremment/decrement a pointer , no matter what type it is, you will have a "cursor to the memory" and can even overflow and see memory beyond what you have allocated.. thats where the nasty bugs come from :)

the type in the pointer will only tell (when you increment) by how much this incrementation will go for each iteration..

So if its a integer pointer you will advance 4 by 4 (in 32 bit) and if its a void* 8 by 8.. .. a small int 2 by 2.. a char .. 1 by 1.. etc..

<comment><USER11> Firstly, a data pointer is allowed to be a different size from a function pointer (although in most platforms they are the same size).

Secondly, what he said is that pointers are not just addresses, but _typed_ addresses that know what type they're pointing at.

<comment><USER10> > Secondly, what he said is that pointers are not just addresses, but typed > addresses that know what type they're pointing at.

Hum Ok! At first i thought he was disagreeing about pointer representing addresses; now with the way you have put it, and by reading again, i see he is _adding_ more information to it.. thats not only address but a _typed_ address (unless void of course)..

Sorry for misunderstanding.. i want to reiterate that i agree with you "dicroce" ;)

</comment></comment></comment></comment><comment><USER12> Over using the pre-processor like this (creating "debug" type macros) always rubs me up the wrong way because of the hassle it is for me to read and ironically, to debug.

There are so many gotchas with writing good quality, safe macros and they're also not well handled by many tools. E.g. my IDE is pretty hopeless at following complex macros.

We've got better alternatives today. Unit testing is probably the first one everyone thinks of but i reckon switching to a graphical debugger (i used to only use CLI gdb) also reduced my need for these kind of hacks.

I feel like this article should be named "C's naughty bits", e.g. the array indexing in #7.

There are some good ones though, i like the idea of better separating code and data per #15.

</comment><comment><AUTHOR> Hi all! I never expected this to reach the front page! Thank you all! This was a quick writeup of a talk I delivered in our college's Open Source Community ([http://pesos.pes.edu](http://pesos.pes.edu)). This was meant for beginners and average C programmers and summarized it through a blog post. I can understand few things do seem a bit different, but the post intends to mainly convey the idea than the exact implementation aspects!

Hope you enjoyed it! Thanks!

</comment><comment><USER13>
      Feature/Trick #4 :

      #ifndef FILE_H
      #define FILE_H
       ...
      #endif

How is this "Cool C Programming"? What am I missing?

<comment><USER8> It might as well talk about #pragma once if it is explaining include guards.

Fun fact: Oracle (Sun) Studio doesn't support #pragma once. Instead, it actually looks for proper include guards in the header and if they exist, it optimizes away re-includes of the file if the condition is false.

<comment><USER14> I assumed that was common practice.

<comment><USER8> Nope, and to make matters worse -- IBM xlC doesn't optimize at the include sites. So adding include-site guards actually speeds up compilation times in a non-insignificant way. e.g.,

      #ifndef FOO_H
      #include <foo.h>
      #endif

</comment></comment></comment></comment><comment><USER15> Not bad reading for a beginner/intermediate C programmer. However, if you are targeting such, try to emphasize more on possible pitfalls, things to avoid and best practices. What first comes to my mind:

1) Name variables in macro differently (e.g. _X instead of just X) and always use them in parentheses.

2) Always initialize pointers, especially in examples for beginners, as segmentation fault while stepping on uninitialized or dangling pointer is one of the most common errors I have seen.

3) Using -Wall compiler directive is also a good idea to see many possible problems you could miss otherwise.

4) Trick #15 is a really bad practice, it's way better to include header file that declares and defines array, something like Gimp or OpenSSL do when you choose to export output as a header file.

5) Just a suggestion: there is a very handy macro for debugging worth mentioning:

        #ifdef DEBUG
        #define DBG(fmt, ...) \
            printf("%s: " fmt, __func__, ## __VA_ARGS__)
        #else
        #define DBG(fmt, ...)
        #endif

</comment><comment><USER16> Might be obvious to some of you, but I use this to check what my macros expand out into to make sure the syntax is valid.

      #include <stdio.h>

      #define atoa(x) #x

      #define mdb(x) printf("%s\n", atoa(x))

      #define swap(type, x, y) do {type temp = *(x); *(x) = *(y); *(y) = temp;} while(0)

      int main(int argc, char *argv[])

      {

        int cat[2] = {33, 44};

        printf("atoa(cat): %s\n", atoa(cat)); // "cat"

        mdb(swap(int, cat, cat+1)); // expands the macro inside to text

        swap(int, cat, cat+1); // the macro itself

        printf("cat[0]: %d\n", cat[0]); // 44

        return 0;
      }

</comment><comment><USER17> When I was in highschool we had this debugging macro:

        #define dbg(...) fprintf(stderr, __VA_ARGS__)

So the dbg function is just a shortcut to "printf to stderr". It also was easy to disable with #ifdef.

<comment><USER10> Did you learn this in high school?

Im curious because kids starting to get by using C is such a nice thing.. cause its a very thin layer for how machines works..

Alienation from the machine inner working(if learning by JS or Python for instance) is such a bad thing, and can take years to the grown up to recover from it :)

Im not against learning the "relaxed" languages, but its better when you do not start with them.. when you only learn them as more tools to do all sort of jobs we have to face

I was lucky to be a teenager at the nineties when programming were more sane and would make us learn how the real machine works.. (before the java madness)

Programmers cannot be lazy, they need to know everything in deep detail, to make the best decisions.. Lazy programmers, very weak programs..

The kids from where you grow up, should be very good when they are adults and decide to work with technology ?! :)

<comment><USER17> We did Topcoder in school, so C was the best option (java wasn't available on local competitions)

</comment></comment></comment><comment><USER18> ...erm... more 'C for newbs'.

Although, it does point out some interesting idioms of the language and I agree that scanf is bloody cool. IMO every language should have printf/scanf equivalents :o)

</comment><comment><USER19> Anything involving the preprocessor ranks high in my favorite C trick (I once got a "clever abuse of the C preprocessor" notation after writing a C+ library for school written entirely using macros).

Actually, anything involving meta-programming is "cool". Knowing a language is "easy" once you get the logic, but manipulating the language to do the work for you is even more satisfying.

</comment><comment><USER20>
      char* first_name; int age;

      fscanf(fptr, "%s %d", first_name, &age);
      printf("Name: %s\nAge: %d\n\n", first_name, age);

Cool usage of uninitialized char* pointer bro.

(Sorry about the snarky tone, I just wanted to use that meme because it's appropriate.)

<comment><AUTHOR> Yeah sorry for the mistake! Will change it soon!

<comment><USER20> It's not your fault, just a good example of the kinds of programming errors this language design makes more likely.

</comment></comment></comment><comment><USER21> Cool tricks!

About the debugging macros: printf prints messages to stdout, not stderr. You want to use fprintf instead:

      fprintf(stderr ,"error message");

This has the advantage of being unbuffered, and still printing to the console even if stdout has been directed to a file.

<comment><AUTHOR> Haha, I started with talking about the standard buffers and have completely missed using them! Thanks for reminding this!

</comment></comment><comment><USER22> I was intrigued by the bit regarding array indexing, but I have yet to _break out vim and mod that C source_. How would this apply to multidimensional arrays?

</comment><comment><USER23> Never seen curly brackets called flower brackets before :) That's new.

<comment><USER24> You'd have if you learned programming in India :)

</comment></comment><comment><USER25> some good tips

although i do think __ FUNCSIG __ deserves a mention along with __FILE__ and __LINE__ :)

<comment><USER26> Ah, except __FUNCSIG__ isn't portable/standard.

<comment><USER25> i believe it works with the gcc, clang and msvs compilers across ios, android, windows phone, linux, windows desktop, windows metro, mac os and the game console compilers.

sure its not standard, but in practice its never caused me a problem.

<comment><USER26> Funny thing is, I tried using __FUNCSIG__ with gcc and clang before I posted. Doesn't work; gcc 4.8.0 on Linux and clang 3.1 on Cygwin. Looks like a Microsoftism to me.

</comment></comment></comment></comment><comment><USER27> Great post! Really loved it!</comment>


Trends, Expectations, and Truth About Web Design 2013 - <AUTHOR> http://www.htmlcut.com/blog/trends-web-design-2013.html

<comment><AUTHOR> This is one of the most comprehensive lists of articles and posts about web design 2013: trends, expectations, the best of 2012 - resources, tools, fonts, inspiration collections.</comment>


Ftfy – fix Unicode that's broken in various ways - <AUTHOR> https://ftfy.now.sh/ <comment><USER2> Ftfy and Unidecode[1] are two of the first libraries I reach for when munging around with a new dataset. My third favorite is the regex filter [^ -~]. If you know you're working with ASCII data, that single regex against any untrusted data is resolves soooo many potential headaches.

I manage and source data for a lead generation/marketing firm, and dealing with free text data from all over the place is a nightmare. Even when working with billion dollar firms, data purchases take the form of CSV files with limited or no documentation on structure or formatting or encoding, sales- oriented data dictionaries, and FTP drops. I have a preprocessing script in python that strips out lines that can't be parsed as utf8, stage it into a Redshift cluster, then hit it with a UDF in Redshift called kitchen_sink that encapsulates a lot of my text cleanup and validation heuristics like the above.

<comment><USER3> This. Ftfy has saved my as many many times. Thanks for the regex. brilliant.

<comment><USER2> I don't remember where I came across that regex, but it's saved me from so many headaches I quite literally get giddy any time I can insert it into a processing stream.

A developer upstream naively split on /n and leave some errant /r characters everywhere? Fixed.

Embedded backspaces or nulls or tabs or any sort of control character? Gone.

Non-ASCII data in a field you _know_ should be ASCII only? Ha, not today good sir!

Until you've had to deal with the hell that is raw, free form data from who- knows-where, you cannot even fathom how satisfying it is to be able to deploy that regex (when appropriate) and know beyond a doubt that you can't have any more gremlins hiding in that particular field/data that'll hit you later.

<comment><USER4> It's nice not having to deal with languages other than English.

In Portuguese, which I've worked with, you develop other tricks, like replacing à, á, â or ã with a. But, in order to do this, you still need to find out the encoding used before you can create the "ASCII" equivalent.

Fun trivia: coco means coconut; cocô means poo. So, by replacing ô with o, you're guaranteed a chuckle at some stage.

<comment><USER5> True. But if that specific case is dealing in, let's say, URLs, then all of the content should be ASCII - either as a direct character representation, or encoded, again to ASCII.

Never did I see the parent mention "this is sufficient for humans", or just "...for English" \- even that would be a naïve assumption (see? ;)).

<comment><USER6> If your definition of "URLs" includes "IRIs" (a term nobody really uses but which encompasses the idea that you can use Unicode in URLs), then this isn't a good assumption to make.

I would rather pass around a link to [https://ja.wikipedia.org/wiki/メインページ](https://ja.wikipedia.org/wiki/メインページ) than to [https://ja.wikipedia.org/wiki/%E3%83%A1%E3%82%A4%E3%83%B3%E3...](https://ja.wikipedia.org/wiki/%E3%83%A1%E3%82%A4%E3%83%B3%E3%83%9A%E3%83%BC%E3%82%B8) , and lots of software will support both. And if you want all URLs to be ASCII, you'll need to convert the first into the second, not just delete the Japanese characters.

<comment><USER7> If the URL is "on the wire" it had better be 7-bit ASCII. Actually even more restrictive than that. Because that's the spec. [https://tools.ietf.org/html/rfc3986](https://tools.ietf.org/html/rfc3986)

In user interaction with a browser or wherever else it seems that anything goes.

<comment><USER6> That's the spec but this is also the spec: [https://www.ietf.org/rfc/rfc3987.txt](https://www.ietf.org/rfc/rfc3987.txt)

It's true that protocols such as HTTP only use ASCII URIs on the wire. If you are implementing an HTTP client yourself, you will need to implement percent- encoding.

Which is different from saying "well, I know URLs should be ASCII on the wire, so I can safely delete all the non-ASCII characters from a URL." That's not true.

<comment><USER5> As soon as you add or remove to the string, it no longer points to the same resource; I considered this too obvious to mention. OTOH, for _validation_, this is useful: "you have a ZWJ character in an URL, that's unlikely". And yes, I understand that there are protocols that allow you to pass around the full Unicode or aunt Matilda or whatever - I should have been more specific.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> \- _Submits issue to Chromium requesting it just run webpages through this before displaying them_ (I've stumbled on old webpages with legitimately broken encoding within the past 4-5 months)

\- _Creates Rube Goldberg machine to fix UTF-8 text copy-pasted through TigerVNC_ (which I was surprised to discover setting LC_ALL doesn't fix)

\--

Fun trivia w/ VNC, because it's cute:

1\. Example from site: #╨┐╤Ç╨╨╨╕╨╗╤î╨╜╨╛╨╡╨┐╨╕╤é╨╨╜╨╕╨╡

2\. Fixed: #правильноепитание

3\. What I get when I copy #2 through VNC: #правильноепитание

4\. What happens when ftfy sees #3: #правильноепитание

5\. What happens when I copy #1 through VNC: #правильноепитание

6\. What happens when I paste #5 to ftfy: #правильноепитание

This is absolutely awesome.

VNC details: Arch Linux server, Slackware client, both running TigerVNC 1.8.0.

Bonus: what happens when I paste the above into Google Translate: "# Proper nutrition" (nice!)

Edit: Wow, Arc didn't choke on this. It has good Unicode support. Nice.

<comment><USER6> But if Chromium ran ftfy on all text, then you wouldn't be able to read the examples on the page, or in your post :(

(In general, I will claim that the only false positives you're likely to encounter using ftfy are test cases for ftfy!)

<comment><USER8> Very good point.

This is an awesome library. My real question is... how on earth does it figure out the text is "okay", that the demangling process is done?

<comment><USER6> Each de-mangling step decreases a "cost" metric on the text, based on its length plus the number of unusual combinations of characters. It never really decides that text is "okay", but when there's no step it can take that decreases the cost, it's done.

This is an imperfect greedy strategy, incidentally. If it takes multiple steps to fix some text, it's possible that the first step it needs to take is not the one that decreases the cost as much as possible, that it has to go through some awful-looking intermediate state so that everything falls into place for the next step. This is rare, though. I don't think I could come up with an example.

<comment><USER8> oooooh. Very interesting.

And I think I understand what you mean.

</comment></comment></comment></comment></comment><comment><USER6> I'm happy to see this Web implementation of ftfy! I especially appreciate how it converts ftfy's fixing steps into example Python code.

Here's an interesting follow-up question for HN: one of the things that makes ftfy work is the "ftfy.bad_codecs" subpackage. It registers new text codecs in Python for encodings Python doesn't support. Should I be looking into actually making this part of Python?

To elaborate: once ftfy detects that text has been decoded in the wrong encoding, it needs to decode it in the right encoding, but that encoding may very well be one that's not built into Python. CESU-8 (a brain-damaged way to layer UTF-8 on top of UTF-16) would be one example. That one, at least, is gradually going away in the wild (I thank emoji for this).

Other examples are the encodings that I've given names of the form "sloppy- windows-NNNN", such as "sloppy-windows-1252". This is where you take a Windows codepage with holes in it, such as the well-known Windows-1252 codepage, and fill the holes with the useless control characters that are there in Latin-1. (Why would you do such a thing? Well, because you get an encoding that's compatible with Windows and that can losslessly round-trip any bytes.)

This has become such common practice on the Web that it's actually been standardized by WHATWG [1].

If a Web page says it's in "latin-1", or "iso-8859-1", or "windows-1252", a modern Web browser will actually decode it as what I've called "sloppy- windows-1252". So perhaps this encoding needs a new name, such as "web- windows-1252" or maybe "whatwg-1252". And similarly for 1251 and all the others.

But instead of just doing this in the ftfy.bad_codecs subpackage, should I be submitting a patch to Python itself to add "web-windows-NNNN" encodings, because Python should be able to decode these now-standardized encodings? Feel free to bikeshed what the encoding name should be, too.

[1] [https://encoding.spec.whatwg.org/#legacy-single-byte- encodin...](https://encoding.spec.whatwg.org/#legacy-single-byte-encodings)

<comment><USER9> Isn't it better staying out? It seems like it shares similar properties to pytz - new codecs need to be added semi-regularly.

What was it Kenneth Reitz said? Something like standard library is where packages go to die.

<comment><USER6> Yeah, I know the saying.

My observation here is that the number of text encodings is generally decreasing, due to the fact that UTF-8 is obviously good. I _want_ wacky encodings to die. But this is just a class of encodings that have existed for decades and that Python missed. Perhaps on the basis that they were non- standard nonsense, but now they're standardized.

It could be argued that web-windows-1252 is the third most common encoding in the world.

If I'm giving directions for how to decode text in this encoding, it currently only works if you've imported ftfy first, even if you don't need ftfy.

<comment><AUTHOR> Sounds to me like you've argued yourself around to pitching them for inclusion! I find the argument that web-windows-1252 is supported by modern browsers very convincing.

</comment></comment></comment></comment><comment><USER10> I'm trying to learn Chinese. I wrote [http://pingtype.github.io](http://pingtype.github.io) to parse blocks of text, and I'm now building up a large data set of movie subtitles, song lyrics, Bible translations, etc.

Try reading this in TextWrangler: 1 . 教會組織: 小會:代議長老郭倍宏

The box causes the follow characters to be unreadable - it gets interpreted as a half-character. Deleting it makes the text show correctly.

I tried it with ftfy, but it just copied the input through to the output.

<comment><USER6> Interesting. Here's the output of ftfy.explain_unicode on that text:

        U+0031  1       [Nd] DIGIT ONE
        U+0020          [Zs] SPACE
        U+002E  .       [Po] FULL STOP
        U+0020          [Zs] SPACE
        U+6559  教      [Lo] CJK UNIFIED IDEOGRAPH-6559
        U+6703  會      [Lo] CJK UNIFIED IDEOGRAPH-6703
        U+7D44  組      [Lo] CJK UNIFIED IDEOGRAPH-7D44
        U+7E54  織      [Lo] CJK UNIFIED IDEOGRAPH-7E54
        U+003A  :       [Po] COLON
        U+0020          [Zs] SPACE
        U+F081  \uf081  [Co] <unknown>
        U+5C0F  小      [Lo] CJK UNIFIED IDEOGRAPH-5C0F
        U+6703  會      [Lo] CJK UNIFIED IDEOGRAPH-6703
        U+003A  :       [Po] COLON
        ...

The anomalous character is U+F081, a character from the Private Use Area. TextWrangler is allowed to interpret it as whatever it wants, but I don't know why that would mess up all the following characters.

Here's my theory. The text probably started out in the GBK encoding (used in mainland China). GBK has had different versions, which supported slightly different sets of characters. A number of these characters (decreasing as both GBK and Unicode updated) have no corresponding character in Unicode, and the standard thing to do when converting them to Unicode has been to convert them into Private Use characters.

So that probably happened to this one, which may have started as a rare and inconsistently-supported character.

Python's implementation of GBK (or GB18030) doesn't know what it is. So maybe what we need to do is flip through this Chinese technical standard [1], or maybe an older version of it, and track down which codepoint was historically mapped to U+F081 and what it is now and hahaha oh god

[1] [https://archive.org/stream/GB18030-2005/GB%2018030-2005#page...](https://archive.org/stream/GB18030-2005/GB%2018030-2005#page/n0/mode/1up)

</comment></comment><comment><USER11> A few months ago I built a simple web interface for ftfy so I don't have to start a Python interpreter whenever I need to decode mangled text: [https://www.linestarve.com/tools/mojibake/](https://www.linestarve.com/tools/mojibake/)

</comment><comment><USER12> Nice. This handles the mangled example I discussed at:

[http://www.pixelbeat.org/docs/unicode_utils/](http://www.pixelbeat.org/docs/unicode_utils/)

</comment><comment><USER13> Reminds me of the Universal Cyrillic decoder [1]

And old MySQL db dump I have has some values such as: !¡!HONDA POW

Does anyone here have an idea if/how I can recover the mangled text?

[1] [https://2cyr.com/decode/?lang=en](https://2cyr.com/decode/?lang=en)

<comment><USER6> In fact, ftfy already figures that text out! Here are the recovery steps that the website outputs:

        import ftfy.bad_codecs  # enables sloppy- codecs
        s = '!¡!HONDA POW'
        s = s.encode('sloppy-windows-1252')
        s = s.decode('utf-8')
        s = s.encode('sloppy-windows-1252')
        s = s.decode('utf-8')
        s = s.encode('latin-1')
        s = s.decode('utf-8')
        print(s)

And the decoded text is (for some reason):

        !¡!HONDA POW

<comment><USER13> Thank you, I'd also tested that but it seems to simply remove the mangled string part. Maybe it's impossible to recover it automatically after all :/

<comment><USER6> No, no. That _is_ the recovered text.

Originally, the text had one non-ASCII character, an upside-down exclamation point. A series of unfortunate (but typical) things happened to that character, turning it into 9 characters of nonsense, the 9th of which is _also_ an upside-down exclamation point.

It looks like ftfy is just removing the first 8 characters, but it's reversing a sequence of very specific things that happened to the text (which just happens to be equivalent to removing the first 8 characters).

</comment></comment></comment></comment><comment><USER14> This is awesome, it reminds me when we decided to add unicode support to our API, but our code had been connecting to MySQL with Latin-1 connection. As long as you read from a Latin-1 connection, it _looked_ like everything was correct, but what was actually being stored was the UTF-8 bytes being decoded as a Latin-1 string, and then re-encoded to UTF-8 since the column was UTF-8. Basically:

string.encode("utf-8").decode("latin-1").encode("utf-8")

although technically what mysql calls latin-1 is actually using Windows-1252 :(

<comment><USER15> _although technically what mysql calls latin-1 is actually using Windows-1252 :(_

...and what mysql calls UTF-8 is a subset that only supports code points of up to three bytes! To get UTF-8 you need to use "utf8mb4". Why anybody uses mysql is beyond me.

</comment></comment><comment><USER16> This could be a useful web service _for interactive use_ , since all strings will be manually verified.

The underlying ftfy library was previously discussed three years ago ([https://news.ycombinator.com/item?id=8187418](https://news.ycombinator.com/item?id=8187418)) and my comments at the time are still relevant:

I can't help but think that this [library] merely gives people the excuse they need for not understanding this "Things-that-are-not-ASCII" problem. Using this library is a desperate attempt to have a just-fix-it function, but it can never cover all cases, and will inevitably corrupt data. To use this library is to remain an ASCII neanderthal, ignorant of the modern world and the difference of text, bytes and encodings.

Let me explain in some detail why this library is not a good thing:

In an ideal world, you would _know_ what encoding bytes are in and could therefore decode them explicitly using the known correct encoding, and this library would be redundant.

If instead, as is often the case in the real world, the coding is _unknown_ , there exists the question of how to resolve the _numerous ambiguities_ which result. A library such as this would have to _guess_ what encoding to use in each specific instance, and the choices it ideally should make are _extremely_ dependent on the circumstances and even the immediate context. As it is, the library is hard-coded with some specific algorithms to choose some encodings over others, and if those assumptions does not match your use case _exactly_ , the library will corrupt your data.

A much better solution would perhaps involve a machine learning solution to the problem, and having the library be trained to deduce the probable encodings from a large set of example data from each user's _individual_ use case. Even these will occasionally be wrong, but at least it would be the best we could do with unknown encodings without resorting to manual processing.

However, a _one-size-fits-all_ "solution" such as this is merely giving people a further excuse to keep not caring about encodings, to pretend that encodings can be "detected", and that there exists such a thing as "plain text".

[…]

I have […] two main arguments:

1\. Due to its simplicity for a large group of naïve users, the library will likely be prone to over- and misuse. Since the library uses guessing as its method of decoding, and by definition a guess may be wrong, this will lead to some unnecessary data corruption in situations where use of this library (and the resulting data corruption) was not actually needed.

2\. The library uses a one-size-fits-all model in the area of guessing encodings and language. This has historically proven to be less than a good idea, since different users in different situations use different data and encodings, and [the] library's algorithm will not fit all situations equally well. I [suggest] that a more tunable and customizable approach would indeed be the best one could do in the cases where the encoding is actually not known. (This minor complexity in use of the library would also have the benefit of discouraging overuse in unwarranted situations, thus also resolving the first point, above.)

<comment><USER6> It's a little strange for you to be criticizing ftfy as an encoding guesser, given that ftfy is not an encoding guesser. Are you thinking of chardet?

> In an ideal world, you would know what encoding bytes are in and could > therefore decode them explicitly using the known correct encoding, and this > library would be redundant.

Twitter is in a known encoding, UTF-8. Most of ftfy's examples come from Twitter. ftfy is not redundant.

When ftfy gets the input "#╨┐╤Ç╨╨╨╕╨╗╤î╨╜╨╛╨╡╨┐╨╕╤é╨╨╜╨╕╨╡", it's not because this tweet was somehow in a different encoding, it's because the bot that tweeted it literally tweeted "#╨┐╤Ç╨╨╨╕╨╗╤î╨╜╨╛╨╡╨┐╨╕╤é╨╨╜╨╕╨╡", in UTF-8, due to _its own_ problems. So you decode the text that was tweeted from UTF-8, and then you start fixing it.

I still think you're thinking of chardet.

> If instead, as is often the case in the real world, the coding is unknown...

...then you will need to detect its encoding somehow. By now ftfy is a library for Python 3 only. If you try to pass bytes into the ftfy function, the Python language itself will stop you.

Are you hypothesizing that everyone dealing with unmarked bytes is passing them through a chain of chardet and ftfy, and blaming ftfy for all the problems that would result?

Incidentally, I do machine learning. (That's _why_ I had to make ftfy, after all.) I have tried many machine learning solutions. They do not come close to ftfy's heuristics, which are designed to have extremely low false positive rates that are not attainable by ML. If you want one false positive per billion inputs... you're going to need like a quadrillion inputs, or you're going to need a lovingly hand-tuned heuristic.

<comment><USER16> > _ftfy is not an encoding guesser_

If it isn't an encoding guesser, what does it do that "".decode("encoding") doesn't do?

<comment><USER17> A guesser answers the question: what encoding did they _actually_ use?

FTFY answers the question: What horrifying sequence of encode/decode transforms could output this sequence of bytes in UTF-8 that, when correctly decoded as UTF-8, still results in total gibberish?

In other words...

The problem fixed by an encoding guesser:

1\. I encode my text with something that's not UTF-8-compatible.

2\. I lie to you and say it's UTF-8.

3\. You decode it as UTF-8 and get nonsense. What the heck?

4\. A guesser tells you what encoding I actually used.

5\. You decode it from the guessed encoding and get text.

      ----

The problem fixed by FTFY:

1\. I encode string S with non-UTF-8 codec C.

2\. I lie that it's UTF-8.

3\. Someone decodes it as UTF-8. It's full of garbage, but they don't care.

4\. They encode that sequence of nonsense symbols, not the original text, as UTF-8. Let's charitably name this "encoding" C'.

5\. They say: Here teddyh, take this nice UTF-8.

6\. You decode it as UTF-8. What the heck?

7\. Is it ISO-8859? Some version of windows-X? Nope. It's UTF-8 carrying C', a non-encoding someone's broken algorithm made up on the spot. There's no decoder that can turn your UTF-8 back into the symbols of S, because the text you got was already garbage.

8\. FTFY figures out what sequence of mismatched encode/decode steps generates text in C' and does the inverse, giving you back C^-1( C'^-1( C'( C( S )))) = S.

</comment></comment></comment></comment><comment><USER18> The mona lisa example is delightful!

</comment><comment><USER19> Damn this is good. I had faced a similar issue where the CSV had mixed encodings. That time I never looked for a library, I read a few SO answers and created an adhoc python script to make the file encoding uniform. Ftfy would have made my work simpler.

</comment><comment><USER20> Nice tool. Just the title is misleading, it's not Unicode that is broken, it's the encoders/decoders..

</comment><comment><USER21> Oh man have I got some spreadsheets I'd love to throw this tool at.

</comment><comment><USER22> for some reason, all examples on that page deal with "sloppy-windows-1252" > "utf-8".

Is it fair to say it is a sloppy-windows-1252 fixer?

<comment><USER23> Not when I click on them they don't.

</comment></comment><comment><USER24> Someone should add this as an automatic preprocessing filter for Slashdot comments.

</comment><comment><USER25> No, Unicode text is not broken. It's either your program that's broken, for it is interpreting ISO-8859-X/Windows-12XX/whatever as UTF-X; or the program that produced said data.

<comment><USER6> I wrote the original library. Your statement is true, but in many cases, not useful.

As quoted from the documentation [1]:

> Of course you're better off if your input is decoded properly and has no > glitches. But you often don't have any control over your input; it's someone > else's mistake, but it's your problem now.

[1] [https://github.com/LuminosoInsight/python- ftfy](https://github.com/LuminosoInsight/python-ftfy)</comment></comment>


Ask HN: How do you handle billing by the day? - <AUTHOR>

Several prominent members of the HN community advocate billing at least by the day for freelancers/consultants. This is extremely appealing to me, and I've begun discussing changing to this billing structure with some of my existing clients.
That being said, I'm still not totally sure how this works. I believe I recall tpacek saying that any client who has any problems or bats an eye at this type of arrangement is pathological and should be avoided. I will acknowledge this is the case, but some of my apparently pathological clients have questions that I don't have really good answers for yet:

  Will you charge me for a day if I have a ten minute phone call/question for you?

  What about 1 hour?

  How will you handle emergencies if you're scheduled to work for someone else that day?

  etc...
 It seems that when a client pays by the day they have a reasonable expectation that you can guarantee that they have your complete attention for the entire day. Earlier this week I actually had my first "by the day" billing engagement on a new project for an existing client. They were scheduled for Monday, and on Monday morning I sat down at my desk to start work for them, only to find a frantic email from a client whose webserver had apparently been hacked. I spent the morning cleaning up that mess, and it was close to noon by the time I got to start working on my supposedly only task for the day. Should I have charged my emergency client for a day's work (assuming I had negotiated that with them, which I hadn't yet) and then charge the client I had scheduled that day for a day's work as well? Or should I have called it a day when I had finished the cleanup, and told the originally scheduled client I would work on their stuff when I had another day free -- in this case, probably not until the new year?
Billing by the day sounds great, but there are a lot of edge cases that I'm not sure how to handle yet. Most of my clients at this point are pretty reasonable to work with, but they are small businesses with non-unlimited budgets. <comment><USER2> Good question. I'm new at it as well. My policy, which I am admittedly experimenting with, is to count a 4+ hour day as one day. I also add up partials and count them as a day when they hit four hours. So:

Monday: 2 hours.

Tuesday: 15 minutes.

Wednesday: 9 hours.

Thursday: 5 hours.

Friday: 3 hours.

adds up to three days plus 1.25 hours accruing. It still has issues like: should this be explained to a client because it's confusing? What do I do about accruals that never reach four? Does it encourage me to work close to four hours per day, which is not the point of it?

Another method I've thought of, which I read on some company's blog, is to assign people to ongoing projects who bill daily, and to also have a sweeper who bills hourly on questions, emergencies and miscellany. I'm not sure if clients would accept such a person's hourly exceeding the daily rate without setting that expectation upfront.

It's hard to fit because the daily method is best for a developer who works to a spec, turns the site over to someone else when done, and always leaves gaps between projects to avoid any overlap due to deployment issues and other problems like that.

And meanwhile, some freelancers I know are saying that weekly is all the rage...

<comment><USER3> the 4+hour model is awesome. thanks for sharing

</comment></comment><comment><USER4> Maybe not all types of contracting work for per day billing.

For me I think it works as I'm paid to deliver a functioning product prototype that then goes to QA. Getting pulled into something urgent that required immediate attention is rare, very rare. So I can pretty easily predict how many hours I can devote a month out.

But in your case, do you have to declare which days you allocate to a customer? I suppose if its sysadmin type stuff you may have a queue that the customer expects you to get to on or by a certain day.

</comment><comment><USER3> I have a friend who bills by the day, because it gives _more_ flexibility and not less. His explanation: sometimes you have days where you only put in 2 hours, other times you have days where you put in 12. He chops a project up into how many "days" he thinks it will take and runs with it.

Really the "days" becomes an arbitrary unit of measurement at that point, which is really what "hours" are too but it's hard to sell that to clients sometimes.</comment>


Amazon.com is selling Uranium Ore - <AUTHOR> http://www.amazon.com/dp/B000796XXM

<comment><USER2> It's so low-radioactivity that it's license exempt. The warning labels are just for novelty.

You probably have a more radioactive americium source in each of your smoke detectors than in that entire jar.

</comment><comment><USER3> Here's another classic. A 55 gallon drum full of personal lubricant: [http://www.amazon.com/Passion-Natural-Water-Based- Lubricant-...](http://www.amazon.com/Passion-Natural-Water-Based-Lubricant- Gallon/dp/B005MR3IVO/ref=sr_1_1?ie=UTF8&qid=1378747813&sr=8-1&keywords)

</comment><comment><USER4> I mean, it's just rocks from the ground right? "Radioactive Ore Sample (NORM) Naturally Occurring Radioactive Materials"

amazon comments are gold though.

</comment><comment><USER5> What is the most interesting thing I could do with this?

<comment><USER6> Make a hardware RNG that's truly random by detecting the interval between decay events :)

<comment><USER7> > detecting the interval between decay events

How do you do that?

<comment><USER6> [http://www.fourmilab.ch/hotbits/hardware3.html](http://www.fourmilab.ch/hotbits/hardware3.html)

HotBits already has implemented the idea

</comment></comment></comment></comment><comment><USER8> If you really want to get on the NSA's list, pick up this Thermite Kit:

[http://www.skylighter.com/mall/product- details.asp?id=2540](http://www.skylighter.com/mall/product- details.asp?id=2540)

<comment><USER9> That is great stuff! :-)

</comment></comment><comment><USER10> I just love how it's categorized under "Occupational Health & Safety Products."

Does this mean I can list my chloroform under the "Cosmetics" category too?

</comment><comment><USER11> The reviews are where the real fun is at.

</comment><comment><USER12> Might I recommend you also add a banana slicer to your cart?

[http://www.amazon.com/Hutzler-571-Banana- Slicer/dp/B0047E0EI...](http://www.amazon.com/Hutzler-571-Banana- Slicer/dp/B0047E0EII/ref=pd_sim_sbs_gro_8)

</comment><comment><USER13> And now I'm probably on some watch list.

</comment><comment><USER14> I'm really enjoying the comments.

</comment><comment><USER15> "I bought this to power a home-made submarine" :P</comment>


How to Learn Any Language in 3 Months - <AUTHOR> http://www.fourhourworkweek.com/blog/2009/01/20/learning-language/ <comment><USER2> What a crock. The first sentence:

 _Principles of cognitive neuroscience and time management can be applied to attain conversational fluency (here defined as 95%+ comprehension and 100% expressive abilities) in 1-3 months._

(a) is bullshit; (b) isn't even given an attempt at substantiation; (c) is barely connected to the gobbledygook of the rest of the post.

</comment><comment><USER3> After such a disappointing article like this I feel a bit stupid that I've just ordered his book (four hour work week) two days ago.

<comment><USER4> I found the book worth the time. His business morals aren't quite what I would strive to emulate, but it's a quick read and there's some interesting stuff in terms of time & money management.

</comment></comment><comment><USER5> A more apt title might be 'How to Learn Enough Japanese to get the Basic Gist of a Martial Arts Book in 90 Days'

This article is all kinds of wrong. You don't learn language by making lists of words. Think of a word like 'but'. In Japanese, there could be 5 different words that 'translate' into 'but'. But you can also use tone of voice or draw out the end of a sentence to convey the meaning of 'but'. In other contexts, you might not want to say 'but' at all.

This all boils down to the author's definition of 'learn' and indeed 'knowing'. If after 90 days, the author is able to say that he 'knows' Japanese, then his bar for 'knowing' is a lot lower than mine.

</comment><comment><USER6> Or move to a country where said language is spoken for 3 months ...

<comment><USER7> I know many, many people who have lived in Japan for 3+ years who struggle at basic conversations, to say nothing of common goals like being able to read a newspaper, function in an office environment, or achieve fluency.

(Cards on the table: I think "fluency" is a poor goal because it is confoundingly difficult to measure. I tend to advice folks to phrase goals for language learning in terms of tasks they cannot currently accomplish which they wish to be able to accomplish in the future. "Am I fluent?" is hard to answer, "Did I understand the Deployment Instructions well enough to actually boot the server?" is easy.)

<comment><USER8> I think Japanese is a bad example.

If you spoke English and moved to Spain for three months, and you spent an hour or more a day actively attempting to learn the language, you would catch on very quickly.

<comment><USER9> These things assume that language learning abilities are roughly universal, and that languages within the same vague groups are roughly as hard as one- another.

German, for instance, turns out to be a lot harder than Spanish, which I'd learned when I was younger. I've been in Germany for 7.5 years, have had girlfriends I only spoke German to, speak German with most of my friends ... and I still struggle with getting the grammar right.

Once again I find Tim's take on things horribly over-simplifying. It seems that he makes rather audacious points by redefining words (e.g. "work", "fluency") and then drawing a bunch of conclusions based on those redefinitions.

<comment><USER10> > German, for instance, turns out to be a lot harder than Spanish, which I'd > learned when I was younger

I'd guess it's the "learned when I was younger" part that made the difference.

<comment><USER11> I studied German when I was younger (middle school/high school), and Italian a bit later (college), and Italian is definitely easier (though no language is _easy_ ).

</comment></comment></comment></comment></comment></comment><comment><USER12> I take his idea of "fluency" with a grain of salt because he hasn't actually posted any videos of him speaking the language.

Fluency is about "nativeness" (if I can create my own word here). It's being able to think in the language (rather than translating), about being able to express yourself in slang/sayings/expressions that you don't learn in textbooks, about the accent you speak in, and so on. In this respect I REALLY doubt he reached fluency.

The reason why many people fail to learn a language is because they use textbooks, which mean well, but are never enough. With Japanese, it always seems to be the same where each chapter is about: \- Vocab list \- A passage to read \- Grammar explanation \- Exercises By the time you finish the chapter you would have understood a bit. The next chapter would do the same, but without refering to EVERY SINGLE vocab you learnt in the last chapter. Repeat this process and you're upto chapter 10 but can't remember most of what you've learnt from chapters 4, 5, 6 and 7.

The reason why living in the country works best is because you're constantly surrounded by everything you're learning. The chapter isn't finished, it's constantly being rehearsed.

<comment><USER13> If you watch some of his "Random" videos with Kevin Rose (especially the ones related to their trip to China), you can hear Tim speak/use Chinese. In Episode 6, he interacts with their waitress and places a customized order.

<http://www.fourhourworkweek.com/blog/category/random/>

I'm impressed with his Chinese speaking abilities. And I'd guess that Chinese is a more difficult spoken language than Japanese since Chinese is a tonal language.

Chinese is my first language, but I was raised in the US. I'd guess within a couple of days my pronunciation would be better than Tim's, but he probably has a better vocabulary.

</comment></comment><comment><USER14> This is complete rubbish. I speak a couple of languages and I seriously doubt even now I could pick up another in 3 months (unless it was very similar to one I currently know).

</comment><comment><USER15> Searching online, I found it interesting that the author supposedly knows 6 different languages and made his fortune when he sold his completely dietary supplement company that he started during the dotcom boom.

<http://www.brainquicken.com/>

<comment><USER15> Typo correction: "completely dietary" -> "completely outsourced dietary"

</comment></comment><comment><USER16> I wonder what the author means by "learning a language". Learning to read simple documents within 3 months isn't that difficult.

<comment><USER17> Although I think everyone should keep a healthy sense of skepticism toward Tim Ferris since he is above all a brilliant self promoter, I must say I was impressed by his command of Mandarin as seen here <http://www.fourhourworkweek.com/blog/category/random/> (Not that he's saying anything especially difficult, but his tones and fluency seem very solid).

</comment></comment><comment><USER18> Am I wrong or does the anti-copy-and-paste on his site tell you something about his zeal to spread the word about his exciting discoveries? Bit petty isn't it?

<comment><USER19> What "anti-copy-and-paste"? I copy and pasted the text of the article into notepad without any problems. I was actually expecting problems since numbered lists frequently get mangled going from HTML to plain text, but even that didn't happen.

</comment></comment><comment><USER20> I don't believe it. Same goes for his 4 hours a month work out. If you are going to make claims like that at least provide details so we can try it.

</comment><comment><USER21> Anyone have any links to develop on this idea?

<comment><USER22> [http://sanbit.com/blog/what-does-it-mean-to-be-fluent-in- a-f...](http://sanbit.com/blog/what-does-it-mean-to-be-fluent-in-a-foreign- language)

</comment></comment><comment><USER23> The languages are different. Try to learn Tibetan, for example, or try to map Japanese writing system to English, without knowledge of how to read Japanese =)

Making and sorting lists is an essence of almost any science, so, making a list of most commonly used words, sort them and then try to memorize will lead you to know some words, but not an language.

<comment><USER24> _map Japanese writing system to English_

 _Don't_ do that. As a polyglot, I recommend you stop transmapping languages; instead, learn each on it's own, mapping it only to your own mental _meaning_. Hard to explain, but it's better to learn grammar implicitly; remember the exceptions (e.g. irregular verbs) individually and whenever you need to conjugate a new verb, go with your gut feeling and see if it's similar to one of the common rules or one of the exceptions you have learned.

For acquiring vocabulary, learn the meaning of the word (internal mental meaning) and its dictionary description in the _new_ language. You will be using the language itself to describe its vocabulary. That way, if you forget the word, you can rewrite the phrase in your mind and expand the description in-place (e.g. "I need to call my father's brother" is still a correct sentence, even if you forget the word "uncle".)

For newbie language learners, it's the single words that are hard to remember, since they might resemble other words, while descriptive phrases are easy to remember since they're long string sequences and whence easy to match in your head.

P.S. Also, if you have a buddy who speaks English (preferably very little of it) try to troll him by generating sentences and let him correct you. It's not as tiring to him as it might seem at first, most people are amused by bad grammar and you will be a town pub favorite. Absorb all the corrections and try to use them immediately, generating other similar sentences.

<comment><USER25> _I recommend you stop transmapping languages; instead, learn each on it's own, mapping it only to your own mental meaning._

Second that. I spent a year in Japan, and got to the point where I could have fairly comfortable conversations, even in both languages if you were at a party with both native speakers. What _really_ stressed out the mental faculties was having to translate, because you had to do the whole mental re- mapping, which you didn't have to go through for a single-language conversation (or even for two separate conversations in different languages).</comment></comment></comment>


Paul Graham is too arrogant - <AUTHOR> http://livinginfirstlife.wordpress.com/2007/05/14/paul-graham-and-y-combinator-true-lies-and-damn-lies/ <comment><USER2> I posted the following comment in defense of my co-founder Wayne:

Jay,

It is supremely unfair of you to characterize my co-founder as irresponsible, selfish, and ruining his family.

I have never met a man who loves his wife more than Wayne. He and his wife reached a decision together that the best thing for their family and their future was for him to come start a company. Your post makes it sound as if he snuck over the fence in the middle of the night.

It is one thing to attack a business, or even to attack the visible leader of said business. But to slander a good hard working father, husband, and friend is criminal. You owe him a public apology.

Sincerely, Robby Walker

EDIT: it appears he listened - the language has been softened a bit and he apologized.

</comment><comment><USER3> Essay comes off as a poorly written rant. However, there is a sliver of truth there; PG would benefit from a little more humility. So would the whole program, actually. It sometimes has a very "clubby" feel to it - you're either in, or you're not.

Even though this isn't a big issue now, it might benefit them to address it before YC gets a reputation of arrogance. This is especially important, given YC essentially bashes a thousand egos a year. Even if 1% of those guys start blogs like this, that's bad news.

<comment><USER4> Can you give me an example of something YC does that seems clubby, and that we could do differently to cure that?

<comment><USER5> The problem is that success makes other people feel bad, and so even neutral statements get interpreted in the worst possible way, and of course it also attracts haters (such as this 'livinginfirstlife' guy), who will repaint everything to look bad.

This was certainly the experience at Google. I don't think that there is necessarily a solution, though it helps to avoid things that could be misinterpreted, even though that also means avoiding some aspects of the truth unfortunately.

<comment><USER6> "The problem is that success makes other people feel bad"

It's more that success attracts attention, some fraction of attention will always be negative, and the haters tend to scream louder than the likers. I'm still a loyal GMail user, I still visit Reddit frequently, and (despite some quibbles and a rejection) I generally think YCombinator's done a lot of good for the startup community. But you don't hear from folks like me often: we're happy, the software does what we want it to, so we have little reason to speak up. Folks with a gripe, however, are willing to put countless hours into venting their spleen.

That was one of the reasons I left FictionAlley (a non-profit Harry Potter fanfiction archive). 99% of users were very appreciative of what we did. However, 1% were not, and the 1% took up about 80% of our time and attention. When you're not being paid for your work (FA was all-volunteer), 80% is far too much time to spend on people with massive entitlement complexes.

(There's perhaps a lesson for prospective founders here: build anything worth visiting, and people _will_ hate you for it. Not a lot, and not nearly as many as it seems, but you better be prepared for it and willing to put it in perspective.)

The latest fad in business now seems to be "fire your customers" - get rid of the 1% that soaks up 80% of your attention, so you can better serve the 99% that actually appreciate what you do. I dunno if that's applicable to YC; probably not, since they "fire" 95% of their customers simply through lack of resources and the haters come from the people who're not doing business with them anyways. Ironic, that is...

</comment></comment></comment></comment><comment><USER7> What I find most amusing about this is guy is how he so passionately bashes Y Combinator on his blog, Techcrunch and probably any other form of medium he can get his hands on. Each time he argues how YC is worthless and irrelevant. The problem being that if YC is so irrelevant and pointless - why spend so much of your time and energy writing about it? I know my general approach to things I think are a waste of time is putting them to the side and spending my time on the things that are in my opinion. Since I value my time as a scarce resource I think that's a justified approach.

The logical assumption then is either:

1) "Livingfirstlife" doesn't actually consider YC to be so pointless as presumably if he genuinely believed that to be true, he wouldn't be so generous as to provide a stream of free PR.

2) He doesn't value his time particularly highly, in which case he's not exactly the type of person I'll be asking for startup advice anytime soon.

Perhaps one day he'll learn that childish tantrums aren't the most effective method for convincing people your point of view is correct.

<comment><USER8> Harj I'm a little confused. If you're someone that YC has invested in, why do you have so much spare time to argue with this guy? His tantrums aren't very childish and I think you're showing your lack of maturity by not presenting any general arguments. You clearly aren't valuing your time very highly either - you're just being hypocritical.

<comment><USER7> I've posted one techcrunch comment and one comment here disagreeing with him. That's a combined total of about 10 minutes of my time which in my view is justified use of it, as it was time spent defending an organization that has done nothing but provide me with a stream of endless opportunity.

</comment></comment></comment><comment><USER9> pg, whatever you read into this rant, please don't let it make you go soft and pull your punches in essays, posts, talks or interviews. Most of your audience prefer your straight talk to mealy mouthed, watered down, PC blandspeak, even when you're wrong. And if destitution necessitated scurvy (out of all things......) turns out to be a huge problem amongst your founders, I suspect there are investors out there willing to come to the rescue, even at twice YC's valuations. No questions asked :).

PS: for those who find the IQ test comment arrogant; Paul Graham is just showing off his mastery of Cambridgeian lingo. Back there, saying someone has low IQ is short for saying they disagree with you. Nothing more, nothing less, and absolutely nothing to be all up in arms about.

<comment><USER6> "Paul Graham is just showing off his mastery of Cambridgeian lingo"

Technically, it's "Canterbridgian". ;-)

</comment></comment><comment><USER10> I guess the Newsweek article is bringing PG a lot of unwanted attention.

\ "there is no need for a marketing budget when you've got Internet word of mouth." Of course - build it and they will come right?

In PG's defense I don't think he meant to imply that if you build it they'll come. I think he was emphasising the power of Internet word of mouth for an Internet business as opposed to traditions marketing channels. BTW. It doesn't seem like the above quote was PG's, unless I'm mistaken.

That being said, the author made some valid points. For one, even if the YC hopefully know they gonna accept the offer even before applying, giving them 5 mins to respond, and then suggesting that anyone not responding within that time in unintelligent makes PG come of as arrogant, if not a bully.

Secondly, I don't think PG can be blamed for a YC founder's personal choice to leave his pregnant wife to pursue a startup (this is a far stretch from the true story) and poor dietary choices of YC founders.

</comment><comment><USER11> Wow,

This guy's whole blog (not just the article) is based on bashing YC & YC funded companies. There are some valid cons to YC Funding: the funding comes at a low valuation, and it is scary to build a company without a clear path to monetization. But this guy basically repeats this over, and over, and over again. Don't expect any enlightenment. Lame.

</comment><comment><USER4> Some comments on this thread were upvoted by several sockpuppet accounts at the same IP addr as infirstlive (including angelasimpson and hotstufff). There was also some sockpuppet downvoting of comments disagreeing with them. Seven of the upvotes on the link itself were from the same source. I've manually fixed the scores in a lot of the cases. Sorry, will add more anti-sockpuppet features in June.

</comment><comment><USER12> I'd rather base my opinion of YCombinator on the opinions of those who have actually gone through the program and have an insight and perspective about the program that are much more objective than any outsider's. If there's anyone who'd claim that joining YCombinator is a stupid idea, it'd be those who were actually put through the process.

However, I do have to say, that being rejected from YC and being an outside observer, I do at times feel that YC itself seems to get a bit 'clubby.' One big reason for this feeling comes straight out of pure envy and jealousy for wanting to be a YC member. The other reason deals with the fact that I feel as if the connections/information gained through such a program are ones that no one on the outside has access to. So it's either you're in and you're connected, or you're out and left in the dark.

<comment><USER6> There were a couple YC members at the recent Boston startup meeting (Tsumoki was there, and I think Dropbox, and possibly one other that I didn't catch the name of)..They didn't seem clubby or standoffish at all. Really, they seemed like pretty approachable, frighteningly intelligent, and generally decent guys.

</comment></comment><comment><USER13> I like Paul. He is one of rare cases when an engineer, besides being good at what he does (engineering), on top of that also has business smarts and CAN WRITE. That is very refreshing after all this MBA BS in the mainstream media.

As an investor, however, PS and Co is not a very good fit for vast majority of engineers because... because they give you only what... 5K per founder? That is much less than good engineer's one month salary after-taxes. That limits his investments to trivial weekend projects completed by fresh college grads. It's the investment side of PG, that the blog post is bitching about.

<comment><USER14> I don't see what the point of complaining is, though. Everyone knows what the terms are, so do your own calculations, and if it's not worth it to you, don't apply.

<comment><USER13> I guess alot of people, including this guy, are openly questioning "validity" of YComb's startups. Since everything they do is so trivial, can their creations be considered "products with value"?

My personal issue with YComb's approach is that they _actively_ (via this site, for instance) promote engineering mediocrity. Remember posts "PG made me drop out of college today!" ? I have seen several of those. Web 2.0 is all about this: cheap&quick; high-school level programming project done over a weekend, polluted with google ads with mandatory "blog" attached to it.

To me that is sad, There is nothing "glamorous" about it. My major in school was AI, and our professor used to dream alot about more powerful hardware that was supposed to become a reality soon. Well, it became. And for what? To run 8 Mongrels with some quick&dirty; Ruby scripts converting contents of 4 DB tables into nicely looking HTML with roundeed corners? What a waste...

<comment><USER15> Business does not equal mastery. if you complain about MBA BS then it should be obvious enough. The iPod was nothing new in terms of technical mastery. But it was innovative in terms of interface design, and a damn good example of successful marketing and integration into a whole product family; as such, it's also an innovation of process.

But does an iPod engineer understand any AI? Probably not. Steve Jobs made stupid comments in his presentations (still remember the "branch prediction... it predicts branches!"). But he has an eye for opportunities that not even your professor can boast having. If there's a market, and you know what it takes to supply it, supply it. In my view, YC is addressing this; it may saturate quickly, but the need obviously exists. If your expertise targets the extremely educated, by all means make something that satisfies them.

But don't forget that there are a lot of people to please. Just please the ones you understand, or think you do. I'm completely taken aback by twitter's success, but hey, people use it, what can I say.

</comment></comment></comment></comment><comment><USER16> I read raves about Yahoo Store long before I ever heard of Viaweb or PG. If a best-of-breed web app used by thousands of businesses is a "crappy start-up", we need more crappy start-ups. I think he forgot an S -- they were _S_ crappy.

</comment><comment><USER17> On a side note, what is the problem at Justin.tv, if there is one? The article mentioned that it is in free fall - really?

Justin.tv page seems to not work in Firefox anymore, so I couldn't see any announcements they might have made on their web page.

</comment><comment><USER18> If he gets rich, they won't call it "arrogance". People would call Steve Jobs arrogant (based on his public statements) if he weren't so damn successful.

</comment><comment><USER19> See also <http://news.ycombinator.com/comments?id=22277> ;)

</comment><comment><USER20> Historically the companies that bragged before they were successful imploded the fastest.

</comment><comment><USER21> this guy killed his credibility by using AliG as an avatar

</comment><comment><USER22> got link bait?

</comment><comment><USER23> Sour grapes, it sounds like.

"Paul Graham & Co. take equity at an extremely low valuation.

Because the company is not worth very much when it's only days or weeks old. So?

"watch Justin.tv and you'll see a business in free-fall"

Huh? What makes him say that? Justin.tv has been live for a few weeks now. Free-fall?

"The number of visitors required to have an advertising driven site that is profitable and viable as a business is staggering"

So not true. You don't have to think very hard to come up with examples (dooce.com comes immediately to mind), but what I really want to suggest is that the post author read The Long Tail. You do _not_ need a huge, mainstream, "staggering" audience to make good money, even with Google AdSense.

"... says another start-up was sold making it seem like a huge deal that closed. Except it's not. That other start-up is called Kiko and it sold for $258,000 on eBay to Tucows who will surely find it was a waste of their own money."

Tucows thinks otherwise, and seems to have pretty good reasoning. See

<http://blog.tucows.com/blog/_archives/2006/9/5/2297315.html>

where they say: "What was the value to Tucows of the time and the certainty? ... What was the value of having it be good for sure? Even if we threw it away in six months (not that we plan to do that)? What I can tell you for certain (and you'll be able to hear more details in an upcoming podcast) is that it was more than we paid!"

"Get Duped. You get 5 minutes to decide."

You should decide when you apply. If you don't know by the time you get the phone call, that's your fault, blogga.

"Ruin Your Health."

If your health is ruined by Lean Cuisine, then you would have died in college when you were subsisting on Top Ramen.

"Hurt your Family."

No one is forcing you to leave your pregnant wife. Hell, no one is even encouraging that.

"Become Pinocchio ... You pretend secrecy is important when you need to cover up a bomb or lack of actual interest from an acquirer."

That would be no different than any other company, except that this post's "evidence" that YC does this is scant and misrepresents the facts.

Blogga, PLEASE.

<comment><USER6> Justin.TV's Alexa rankings have been tanking, to the point where ustream.tv has almost overtaken them:

<http://alexa.com/data/details/traffic_details?url=justin.tv>

Granted, that's just Alexa, and Compete.com still shows justin.tv growing healthily (though ustream's growth rate is higher). But still...were I justin.tv, I'd be concerned.

<comment><USER23> Ok, fair enough -- but the site is still only a few weeks old, and the way I understand it, watching Justin's life through his hat-cam is not the whole point of the business model. There is more to come.

</comment></comment></comment><comment><USER8> The blog is a bit of a rant but there are a lot of valid points. I doubt this is someone who got turned down by Y Combinator though since he/she knows nothing about the application process.

That being said, I think the arrogance is reaching all time highs and Paul Graham needs to get it under control.

<comment><USER17> People just don't like to hear the truth, and then they try to avoid listening by just calling it "arrogance". I think PG is just taking some short cuts in his reasoning here and there, and people who can't follow get frustrated by that.

<comment><USER8> I'm still kind of confused. Y Combinator has just one success - Reddit. Everything else has either failed or is To Be Decided...

<comment><USER17> I don't know anything about the numbers, to be honest. It just seems like a very interesting and inspiring environment. I think it would be worth to pay money to be a part of it. If money was the only thing that would interest me, I would probably have become a financial consultant or an estate agent.

</comment></comment></comment></comment><comment><USER24> It's pretty obvious what Paul Grahams goal is. He wants to pay some kids a laughable amount of money to write code for an idea he likes. He then uses his popularity to promote those websites. He basically gets coders for cheap. He's not stupid or anything, it's just he gets the better end of the deal than the kids he gives $5k to.

If you have an idea that paul graham likes, you're better off not taking the money.</comment>


Taking the wrong lesson from Uber - <AUTHOR> https://medium.com/@sarahtavel/taking-the-wrong-lesson-from-uber-ae4b41e7c7da <comment><USER2> Does 10–40% cheaper = 10x better product? plus adding an easy to use interface?</comment>


Ask HN: Which Tech CEOs should Obama Actually have met with - <AUTHOR>

I am pretty disappointed with a lot of the people the President met with, I don't think they reflect technology or our collective knowledge about how to build it.  Just curious who do you think he should have met with.  Both for help with healthcare.gov and a better understanding of the NSA situation. <comment><USER2> A little context would be nice.</comment>


There's a Dangerous Racial Bias in the Body Mass Index - <AUTHOR> https://www.newsweek.com/2017/05/19/obesity-childhood-obesity-body-mass-index-bmi-weight-weight-gain-health-595625.html <comment><USER2> Ugh. It's not "bias", it's simply new science. Or perhaps "new-ish", since most of this has been known for decades.

<comment><AUTHOR> Yes, it is indeed a "bias". Otherwise BMI recommendations would have already been accounting for race.

They account for height and weight, but where do you see mention of race here, for instance? [https://www.nhlbi.nih.gov/health/educational/lose_wt/BMI/bmi...](https://www.nhlbi.nih.gov/health/educational/lose_wt/BMI/bmicalc.htm)

Their recommendations are utterly useless -- even harmful -- to South Asians.</comment></comment>


Ask HN: How do people pair code/debug remotely - <AUTHOR> When trying to help coworkers or friends with a issue/bug, and you are not right beside each other, do you often: share small code snippets, use screen share, remote desktop software or other methods? <comment><USER2> Teletype with an audio call works well for me: [https://teletype.atom.io/](https://teletype.atom.io/)</comment>