Twitter Mood Predicts The Stock Market - <AUTHOR> http://www.technologyreview.com/blog/arxiv/25900/ <comment><USER3> Hey, why stop at passive predicting - if we set up a few boxes with a hundred thousand twitter bots pushing "calmness" updates, we can probably make the Dow keep on going up forever (or at least 87.5% of the time) . Think about all the happy 401(k)s.

<comment><USER4> Cargo Cult Science :D

<comment><USER5> If the markets were actually science rather than the decision making of a few huge players - if the HFT algo writers take this research seriously, I bet it'd work for at least a day or two.

</comment></comment></comment><comment><USER6> From the top right of page 6:

        To assess the statistical significance of the SOFNN achieving
        the above mentioned accuracy of 87.6% in predicting the up and down
        movement of the DJIA we calculate the odds of this result occurring
        by chance. The binomial distribution in- dicates that the probability
        of achieving exactly 87.6% correct guesses over 15 trials (20 days
        minus weekends) with a 50% chance of success on each single trial
        equals 0.32%. Taken over the entire length of our data set (February
        28 to December 20, excluding weekends) we find approximately 10.9
        of such 20 day periods. The odds that the mentioned probability
        would hold by chance for a random period of 20 days within that
        period is then estimated to be 1−(1−0.0032)10.9 = 0.0343 or
        3.4%. The SOFNN direction accuracy is thus most likely not the result
        of chance nor our selecting a specifically favorable test period.

I'm not sure about the result not being chance. In particular, aren't they privileging the hypothesis by using the probability of 87.6% guesses being correct? They say the result only had a 3.4% chance of happening, yet we could assign a similar probability to a wide variety of other seemingly unlikely outcomes. I'd much rather have seen the predictor tuned on a subset of the data and then tested on the rest of the data - perhaps the authors will release their predictor and we can try it on 2009.

Given that twitter and the stock market are both correlated to what is happening in the world (in fact, caused by what is happening in the world,) it is not surprising that they are correlated with each other. Somehow I don't think correlation in the other direction would get the upvotes though - <USER2> market crash predicts sadness on twitter? You don't say.

In short, I don't follow all of their statistics, but I am highly dubious of their model having any predictive power.

<comment><USER7> They managed to test and train on different periods of time.

        February 28, 2008 to November 28, 2008 is
        chosen as the longest possible training period while
        Dec 1 to Dec 19, 2008 was chosen as the test period

That said, their split exhibits enormous bias:

        Dec 1 to Dec 19, 2008 was chosen as the test period
        because it was characterized by stabilization of DJIA
        values after considerable volatility in previous months
        and the absence of any unusual or significant socio
        cultural events

I don't understand why they would think this is okay.

<comment><USER6> Whoops, missed that. Well then, in the absence of unpredictable events, twitter could predict the stock market, but it can't anymore since the market will now factor this new knowledge in (assuming perfect markets of course.)

<comment><USER8> Pendantic response:

Perfectly _efficient_ markets. Which, as a theory, has been under rather serious siege for the better part of the last decade.

You can certainly make a behavioral argument that would reach more or less the same conclusion, however.

</comment></comment></comment></comment><comment><USER9> I am very angry about this. I was damned close to going live. My code was written. My backtesting showing amazing results.

Now every douchenozzle on earth will be onboard the bandwagon, and render it useless.

EVERY TIME I HAVE A GOOD IDEA SOMEONE BEATS ME TO IT! WHARBLEGLARBLE!

<comment><USER10> Do you have any idea what you would have done after your project went live?

Because you can still beat them to that.

<comment><USER9> Yes. I do. I was going to bet on the supershort, and superlong funds. (2-4 x average market returns) for short term periods, and double my money every few days. I was only waiting to build capital.

Now I have zero time to build capital before it's leveraged away. FUDGE!

I am righteously pissed off now. I had investors on the hook. now it will all be arbitraged away from me.

It's just like quicksilverscreen when Rupert Murdoch litigated it away (one of the senators who voted for the DMCA was one of the lawyers defending Fox).

I am ANGRY.

<comment><USER9> I have so damned many phonecalls to make tomorrow. A lot of powerful, and wealthy people will never talk to me again because of this. Fuck. Fuck. Fuckity. Fuck.

<comment><USER11> You may be mad, but unfortunately this is not the place for it. Let us know if it pans out.

<comment><USER9> I just lost billions of potential dollars, because some college kid couldn't see the practical applications and published. What would your response be?

What if you were sleeping in someone else's garage, it was less than freezing temperature outside, someone had stolen your car a few weeks ago, and you had put your own kneecaps on the line as collateral? What would it be then?

Frankly, I think I'm holding together rather well given my circumstances.

<comment><USER12> Relax, dude, I've heard exactly this idea pitched by 2 other people, you'd have been up against "douchenozzles" even if this "douchenozzle" hadn't published.

<comment><USER10> I think this further illustrates the theory put forth that ideas are worthless, and it's only implementations that matter.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER13> Interesting hypothesis but I will wait before trusting my savings to this approach. If twitter mood can be correlated with stock market trends, I don't see how it could fare better than traditional Socioeconomics indices which measure mood in a much more controlled and accurate way.

Interpreting mood trends from Twitter data assumes some very advanced semantic and linguistic analysis of messages, which even my human brain fails to understand about 25% of the time.

</comment><comment><USER14> I still prefer the hem line indicator. ;)

<comment><USER15> <http://en.wikipedia.org/wiki/Hemline_index>

</comment></comment><comment><USER5> My hypothesis: This is just another indication that stock market speculation prices spend most of the time almost completely decoupled from the value of the underlying assets. Eventually there is some triggering event, lets just call it a Lehmanbrovent, when some massively leveraged party can't pay off a creditor, and the effect cascades through the market bringing it back to a real valuation. We call this state a "catastrophe." The decoupled state we refer to as a "healthy market."

A good way to test it would be to see whether the accuracy of the twitter indicator varies based on types of market activities, such as: does it completely fail on crashes, but always predict run-ups?

</comment><comment><USER16> Does anybody know how they got this data?

Does twitter's API offer the ability to see _all_ incoming tweets? Glancing at their API documentation seems to suggest that they turned this ability off a while ago.

<comment><USER17> You can get a pretty big, random chunk of the public timeline with statuses/firehose or statuses/sample

<http://dev.twitter.com/pages/streaming_api_methods>

</comment></comment><comment><USER18> From the paper-

 _Third, these results are strongly indicative of a predictive correlation between measurements of the public mood states from Twitter feeds, but offer no information on the causative mechanisms that may connect public mood states with DJIA values in this manner._

</comment><comment><USER19> I rather like Eric Gilbert's "Widespread Worry and the Stock Market" on this.

<http://social.cs.uiuc.edu/people/gilbert/38>

</comment><comment><USER20> Anyone want to open a hedge fund?

</comment><comment><USER21> Trading != Investing

</comment><comment><USER22> i for one will be investing heavily in tila tequila

<comment><USER23> Invest in synthesizing the next Bieber.

</comment></comment><comment><USER24> I wonder how long it takes for the observation to influence the results, thereby invalidating the observation.</comment>


Sparklyr – R interface for Apache Spark - <AUTHOR> https://blog.rstudio.org/2016/09/27/sparklyr-r-interface-for-apache-spark/ <comment><USER2> This is such an awesome way to deal with data and I can see this making R more prevalent in working with Spark.</comment>


Do Men Really Make More Money Than Women? - <AUTHOR> http://www.payscale.com/gender-lifetime-earnings-gap?r=1#

<comment><USER2> The President of the United States of America, Barack Obama said on TV that women earn 23% less than men for the exact same job... How dare this study say otherwise

</comment><comment><USER3> Repost - [https://news.ycombinator.com/item?id=9427468](https://news.ycombinator.com/item?id=9427468)</comment>


Show HN: Itinerary matching service (carpooling) - <AUTHOR> http://demo.companycarpool.com/ <comment><AUTHOR> Hello, creator here. Do you guys have any suggestions on how to make this product successful? I am trying to sell this service to businesses, in order to offer it to their employees, but it has not been successful so far. Should I write an API? I am also trying to sell it as white-label software. I think there are many cases where such a simple tool can help millions of people doing carpooling efficiently, but struggling to get it in the hands of people, even not making much money.</comment>


Squarespace raises $38.5M from Index Ventures and Accel Partners - <AUTHOR> http://blog.squarespace.com/blog/2010/7/13/index-ventures-and-accel-partners-invest-in-squarespace.html

<comment><USER2> Square space is a website building platform with included hosting.. it appears their going to charge $30/month for a plan that allow people to build sites that include the functionality of collect data (forms)

$8/mo gives you 1 gb with 75gb bandwidth and you use their sub domain

$14/mo give access to having up to "250 users" (email subscription?) and custom URL

$20/mo , $30/mo and $50 really just up the futures to any feature you could create yourself with the $20/month package that includes HTML injection points..

a mash up of woofoo, website builder and hosting...sold to people who probably will never need the 75gb of bandwidth of the first plan..

<comment><USER3> and to people that don't really need web content management with fancy forms only something like a forms builder/hosting and an app that will process and track gathered data (forms + CRM)...

</comment></comment><comment><USER4> I bet the founder took money off the table. Say a $40mil on a $40mil pre, and the founder walks away with $20mil in his pocket. 9mo later they go freemium.

</comment><comment><USER5> Jeez. The offer must have been really attractive.. otherwise why would Squarespace need all that money? Don't say servers.

Conversely, why would VC's throw in so much money into this business? I just can't see the exit strategy. Who can you name off top of your head that could potentially acquire them at a few multiples? I can't.

[Edit] This is just a web design operation, only difference is that the customers get the CMS login. hehe

<comment><USER6> It does seem like a lot of money ... but Automattic has raised over $30M for (primarily) WordPress. And SquareSpace is actually charging money.

Exits? Inuit paid $170M for Homestead in Nov. '07. I'm sure the acquisition climate is different now but I think there are enough potential fits around. Adobe comes to mind seeing as they need a better HTML(5) web solution. Not sure if that's a route they want to go, though.

<comment><USER7> Rackspace? GoDaddy?

<comment><USER8> Rackspace isn't in the business of consumer products. They are slowly becoming middleware for MSPs and a platform for cloud based services.

GoDaddy... now there's something.

</comment></comment></comment></comment><comment><USER9> Carbonmade's offices are two floors above Squarespace's in SoHo, New York and I've gotten to know Anthony (founder) and Dane (CEO) rather well over the past year. They're such great, smart, and talented people. They really know how to run an awesome team and only hire the best people. They've got a great culture in place and always treat their employees very well.

The thing I respect most about Squarespace -- and Anthony mentions this briefly in his letter -- is that they're always one step ahead of everyone else and aren't afraid to take the risk in pivoting into a new direction. Squarespace dropped free accounts from its service about 3 years ago when everyone else was adding free accounts and that clearly was beneficial for them. I'm excited to see what they come up with in their next release.

<comment><USER10> Hell yeah... I don't have the benefit of having offices this close to Squarespace, but after spending just one (awesome) lunch with Anthony & Dane I too can attest that they are super cool & extremely clever. :)

</comment></comment><comment><USER11> I recently started up a new blog (<http://nextended.com> for Sony NEX users) and searched around the available platforms. I've used blogger in the past but it's pretty clunky these days. I still use self-hosted Wordpress for my personal blog but I've always been annoyed by the post editor, less-than- simple customization and frequent security breaches.

In short, I now pay SquareSpace $12/mo to look after hosting, a very attractive and useful CMS, some good looking templates, point-and-click CSS customisations, drag-and-drop layout, and built-in analytics. You could do the same with Wordpress (and, in fairness, customise it more) but SquareSpace wins if you value your time.

<comment><USER8> I used a custom PHP framework for client sites and Wordpress a lot. However, due to my time restraints and their money restraints, a lot of clients have been migrating to Squarespace. This is good because I can still consult and train them, while getting more bang for everyone's buck.

I don't want to manage entire websites on a client-basis anymore. Its one step from MSP IT (not like its a bad thing, I just like code).

</comment></comment><comment><USER12> I can't help but wonder what their BHAG is for all this money. It seems like it's got to be an awfully big target.

<comment><USER13> What does BHAG mean?

<comment><USER14> let me...

er, <http://en.wikipedia.org/wiki/Big_Hairy_Audacious_Goal>

</comment></comment></comment><comment><USER15> I've used Squarespace for the past 2+ months for my blog on NoSQL and it's been great. I looked around at the various options when I thought about starting up and in the end went with Squarespace because of two reasons:

1) It's the hosting provider of highscalability.com and Todd Hoff had a lot of good things to say about the platform. 2) I saw it as the quickest way to get something that looked good and allowed me to just start on the content.

It's not the perfect platform. Sometimes you have to some crazy things to make things work. But overall I've been a very satisfied customer.

If you are interested you can try it out for free for 14-days to see if it works for you.

Derek

</comment><comment><USER16> Squarespace are offering a great service. Is it a bad sign for my aspirations as an entrepreneur though that I don't know how I would spend that much money on a business like that?

I'm guessing a lot would go on infrastructure if they do the hosting themselves?

<comment><USER17> Sales sales sales. And maybe acquisitions.

<comment><USER16> Yeah, I'm guessing they could end up surpassing all to become the CMS of choice for large corporations with their awesome in browser editor.</comment></comment></comment>


Making women coders, one wearable at a time - <AUTHOR> http://sdtimes.com/making-women-coders-one-wearable-time/ <comment><USER2> [http://i.imgur.com/EI6w3zs.jpg](http://i.imgur.com/EI6w3zs.jpg) <\-- I don't know, I only got about 30 seconds to start reading.</comment>


Emacs Lisp's Future - <AUTHOR> https://lists.gnu.org/archive/html/emacs-devel/2014-09/msg00434.html <comment><USER2> If it isn't broken, don't try to fix it.

I am an old guy (professional Lisp developer since about 1981) and my age probably affects my opinion:

It would be a disaster to mess up the Emacs ecosystem. I don't think that Emacs/elisp runs slowly and since I have to use so many different programming languages anyway, needing to know a little elisp is no problem.

I don't care if elisp is not a modern language.

Way off topic, something that I have written before about: think forwards several hundred years. What will the software landscape look like? My bet is that there will be many ancient software systems that have been debugged to near absolute stability over the centuries. Sure new software will be written, but I bet there will be many very old and stable systems that will see little change.

<comment><USER3> Wouldn't it be nice when Emacs would not be blocked when some Lisp routine runs?

Lots of people have written excellent code for GNU Emacs, but the implementation runtime hasn't improved that much.

<comment><USER4> _Wouldn 't it be nice when Emacs would not be blocked when some Lisp routine runs?_

Yes, it would be nice, but can you imagine the bugs that will happen while everyone works out the details of how to do it properly?

<comment><USER5> Even in the current pre-alpha/alpha stage of Guile-Emacs, I can launch a thread from Scheme, do work, return to the main thread, then call an Elisp function on my result. (Scheme and Elisp data types are unified.) No bugs or unpleasant details there.

Calling Elisp functions and accessing Emacs data types (buffers, windows) from multiple threads is another issue; if you don't want to bother with it then don't; you still get all the other benefits of being on Guile. (Calling to _any_ Guile module agnostically as if it were an Elisp library (including, say, Guile's OpenGL module), having an FFI, getting JIT or AOT native compilation in the future, etc.)

</comment></comment></comment></comment><comment><USER6> Given all the reworks of software projects that have a) been started b) then delayed c) then delayed more, "because this time we'll make it perfect" and d) then failed to gather traction because of incompatibilities, it's my humble guess that the only way would be to continue ELisp as it is and slowly rework the compiler/interpreter and the runtime into a more modern codebase, and add missing features such as thread/process based concurrency. Guile, Clojure, Common Lisp or whatever language are going to basically restart Emacs from scratch, no matter how well they're coated with compatibility macros.

It's better to create slight incompatibilities gradually (so that packages can catch up in some humane timeframe). I could imagine the ELisp runtime being intertwined with the Emacs C code badly enough that making radical changes to it would seem "impossible", just like in the CPython codebase it is considered impossible to ditch GIL and modernise the code. But those kind of comments are often excuses because people are lazy and it's nicer to write new code.

<comment><USER7> If I was them, I'd gradually introduce the minimum set of breaking changes to ELisp that converge it on Common Lisp. Even if those changes were unhelpful or removed features. Then eventually, when it maps 1:1 onto a subset of CL, just swap the implementation for a supported CL.

<comment><USER8> Some years ago the a clisp maintainer actually made a minimal demo of Emacs hosted on clisp with just a compatibility layer for Elisp via macros/functions, which seemed to work pretty well. But it didn't progress any further because Stallman vetoed basing Emacs on Common Lisp (he considers CL to be too big a language, and to some extent just doesn't like it). In retrospect I'm not sure this was a good call: Guile as a general VM with custom infrastructure to support both Scheme and Elisp is not really a more clean, minimalistic infrastructure anyway. The semantics of Elisp/Scheme are too different to be able to just stick in a Scheme implementation with a smallish compatibility layer, like you could with Elisp hosted on CL.

<comment><USER7> Too often it seems RMS is why GNU can't have nice things.

<comment><USER9> RMS would never let practicality get in the way of ideology.

The GNU/RMS is best because you're free! (As long as your computing needs are a text editor, compiler, and some unix system utils)

<comment><USER10> That's why I believe their FFI plans are doomed.

We implemented FFI's for emacs over the last decades, but RMS always strongly opposed it, because you could call windows DLL's then. gtk-emacs e.g.

<comment><USER11> RMS is fine with an FFI as long as you cannot use it to inject non-free code. This is usually done by requiring an exported symbol saying that the code is GPL compatible. GCC does it this way:

[https://gcc.gnu.org/onlinedocs/gccint/Plugin- API.html](https://gcc.gnu.org/onlinedocs/gccint/Plugin-API.html)

<comment><USER10> But you cannot call that a FFI then. A FFI is the abbrevation for "Foreign Function Interface", not "Friendly Function Interface". FFI's are not sugar coated.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> I started a project to port Emacs (the C parts) manually to Clojure, with the Emacs Lisp automatically transpiled into Clojure in 2012, most of the work was done in early 2013: [https://github.com/hraberg/deuce](https://github.com/hraberg/deuce)

It can boot Emacs on the JVM and take simple commands, but not much more. It's my intention to eventually revisit this project, but not sure when.

</comment><comment><USER13> "Lack of some features, most notably FFI and concurrency."

Cool -- my '90s hacks are in demand! I added threading support to Guile and wrote libffi back in the 90s.

<comment><USER14> It seems your libffi has been used in Guile since 1.9.8, so does that mean Guile does support FFI and concurrency?

<comment><USER15> Guile does support FFI and concurrency.

</comment></comment></comment><comment><USER16> I've been bringing the following thought up inside various threads but I think it needs to be said in separate one, since I see many people here and elsewhere missing this about Emacs.

Emacs is _not_ a _scriptable editor_. You don't "script it" or "write plugins for it" in a classical sense of those terms. You reprogram, extend and augment a piece of running code on the fly. Seems similar, but feels different. Emacs is a live Lisp image with text-editing features bolted on top. It's basically backwards of how a typical editor/IDE is implemented.

Therefore, it matters what language is used as a base. People mentioned Lua or JavaScript, but they are nowhere near useful enough for the task.

Therefore, it feels to me - and I believe to many other Emacs users as well - it matters that there should be _one base language_. Emacs as an Elisp system with text editing capabilities _feels_ like a whole. Everything fits together nicely and interacts with each other. It is elegant. Aside for inviting maintenance upkeep and general chaos, making Emacs "run" multiple languages at the core is sort of like shattering its soul into many pieces. I don't want to have an editor with multiple-personality disorder.

Imagine you're writing an executable in three different programming languages mixed together at the same time. That class is written in Common Lisp, but it's child classes are written in C++. And exception handling everywhere is written in Python.

The sheer mental effort to make all of these work in a conceptual harmony inside a single program would be enormous. And it would still feel weird.

That is what multiple-extension-language Emacs would feel.

<comment><USER3> > It's basically backwards of how a typical editor/IDE is implemented.

The main difference is that the implementation language is a dynamic language, which is also mostly the implementation language.

That's similar to how some other IDEs work like Smalltalk or Clozure CL on the Mac. But those are not focused on implementing an extensible editor. Those are IDEs with editing features.

</comment></comment><comment><USER17> If emacs supported new languages other than elisp, a lot of new blood will join the community. Once we get the new hackers, they will try and find ways to modify the old codebase. Emacs is a wonderful editor, but to a newcomer it can be a lot daunting.

For example, I have completed the emacs tutorial, I've even read the infos info tutorial and several info pages regarding emacs and still don't know much about ELisp. I downloaded the official emacs manual, which is more than 1000 pdf pages and plan to read it in the next 10 years.

I love the editor, I want to continue to program in Emacs but Kate seems to be much simpler, elegant and quite functional.

<comment><USER18> Does emacs want the kind of developers who cannot be bothered to learn Elisp? Apart from the syntax, Javascript and Lisp are not really so different.

<comment><USER16> They differ in semantics and elegance, and for that reason alone I wouldn't want Emacs to end up scripted in JavaScript.

Sure it would bring new blood. It will also drive away old one; people, for whom Emacs is a Lisp runtime with a text editor bolted on top of it. Switching Emacs from Elisp to something else feels like selling its soul. Adding parallel support for JavaScript and bunch of other languages feels like turning it into a Frankenstein monster.

<comment><USER19> > Sure it would bring new blood. It will also drive away old one; people, for > whom Emacs is a Lisp runtime with a text editor bolted on top of it. > Switching Emacs from Elisp to something else feels like selling its soul.

While I agree with most of your statement, having Emacs switch/transition/support another, more modern form of Lisp like CL surely wouldn't count as selling your soul.

On the contrary you would stay true to your ideals (a Lisp-runtime with an editor) except now the Lisp runtime just got a lot better.

I like Emacs as much as the next guy, and I have dabbled some in Elisp. I can make due. But I sure as hell wouldn't mind getting some of that neat stuff the CL-guys are getting.

<comment><USER16> > _While I agree with most of your statement, having Emacs switch > /transition/support another, more modern form of Lisp like CL surely > wouldn't count as selling your soul._

I... I'm not sure, but I'm leaning towards agreeing. I can tell you that I _don 't get_ those "we're lobotomizing it" vibes when I think about switching Elisp for CL. Maybe because those two look alike and have similar philosophy. But even a thought of switching to Scheme, which is still a Lisp, sends chills down my spine.

I know, this is completely irrational. But maybe this is a part of Emacs's appeal. You get attached to it, down to its very core.

</comment></comment></comment></comment></comment><comment><USER20> Tom Tromey started an automated rewrite of Emacs in Common Lisp in 2012, see the announcement at: [http://tromey.com/blog/?p=709](http://tromey.com/blog/?p=709)

I couldn't find any progress report after [http://tromey.com/blog/?p=751](http://tromey.com/blog/?p=751) so I guess the project has been abandoned early.

<comment><USER21> I think this is the most realistic approach mentioned here, and tromey is actually familiar with the Emacs internals.

<comment><USER5> So is BT Templeton, the person who has been hacking away at Guile-Emacs GSoC after GSoC. :-)

(They seem very skilled, don't be fooled by the fact that it's GSoC; no idea what the skill level of people who take part in that usually tends to be.)

</comment></comment></comment><comment><USER22> I've got the impression that there is more Common Lisp code and programmers around than guile.

<comment><USER23> Literally dozens more.

Look, neither Common Lisp nor any Scheme implementation are exactly topping the charts in popularity these days. This should be about the technical merits of the choice, because I'm pretty sure Emacs Lisp both has more code and more programmers (both represented in the forms of small scripts we've shoved into _scratch_ and evaluated) than both.

<comment> muuh-gnu The decision to go either Common Lisp or Scheme was made based on RMS very personal reasons and conflicts with close-source Lisp machine companies during the first half of the 80s.

GNU has had two common Lisp implementations under their umbrella at least since mid 90s (CLISP (of Viaweb fame) and GCL (of Maxima fame)), but due to RMS' hate of Symbolics and Common Lisp, all of the effort was invested into making Guile run properly (which took them 25 years and still nobody is using it) and the two Lisps were basically orphaned.

The latest effort of guile-ifying Emacs is more pushed by the Guile guys, in order to attach themselves to an already existing killer app (emacs) and to basically force more and more people to use Scheme, because nobody is doing it voluntarily.

The Emacs guys, as you see from the post, are not _really_ excited about the idea of switching to another _language_. They want FFI, multithreading, etc, but they do not necessarily want Scheme. Actually they dont want Scheme at all, adopting Scheme would be just a price to get FFI, etc.

Of course the proper way would be to go Common Lisp. Elisp is and has always been a subset of Common Lisp, and they've had a Common Lisp compatibility layer for decades, and a lot of packages use it. The problem is simply highly political. You'd have to confront RMS, and you'd have to finally get rid of Guile's "official GNU extension language" status.

<comment><USER24> I've been programming with ELisp, CL and Scheme for a long time. Somehow Guile feels like an odd choice. Why not Chicken scheme, or Racket, or Bigloo; which are the top scheme environments you can choose from?

AFAIK, Guile is not used much beyond a few random projects. Maybe I just don't know Guile, really. Maybe it has a decent implementation afterall.

However, I would have gone CL. There are several "emacs-like" editors implemented in CL:

[http://www.cliki.net/cl-emacs](http://www.cliki.net/cl-emacs)

I would argue that implementing ELisp support in Climacs is actually easier, so we can run all the elisp stuff we already have, and slowly move on.

<comment><USER25> Guile was _designed_ as an extension language, which CL was not. So, that's one reason. Guile also has a something like 90% working implementation of an elisp compiler to guile bytecode, a JavaScript implementation and (I think, though may be wrong) a working Lua compiler. None of the other mentioned scheme implementations were designed for embedding, even if they are fine implementations themselves (which they are).

<comment><USER26> If we consider "designed for embedding" as a driving point we can always take ECL (embeddable/extensible common lisp) as a base and get both very nice ffi and threading while still being CL.

</comment></comment></comment></comment></comment></comment><comment><USER27> This problem begs to think larger. Instead of starting with "we are missing these two critical features that the rest of the world has expected as standard since the dinosaurs perished", it might be interesting to ask "what sort of emacs language features would make it the best in the world!"

It is far easier to get a minimum product out when the eventual goal is the stars.

<comment><USER9> I believe this kind of thinking is what led to the massive success that is emacs in the first place.

</comment></comment><comment><USER28> As I began really using Emacs about a year ago, I got the idea that the Emacs code base could actually be a foundation for a computer science curriculum - The Emacs code base touches on scripting, functional programming, application programming, systems programming in C, real-time problems, usability, cross- platform development, etc. etc.

An alliance with Racket would be an interesting option from the standpoint of language communities.

         - The community's core is [ relatively ] vibrant.

         - The community has more than a passing interest in
           developing development tools.

         - The community is exceptionally stable because
           its members are often linked to academia.

         - It copes with endless September really well.

         - Its vision of open resources overlaps GNU
           somewhat.

         - Programming language problems are right up
           the Racket community's alley.

</comment><comment><USER29> I sent a note about using an elisp-on-llvm solution. Someone had done the work already, it appears: [https://github.com/boostpro/emacs-llvm- jit](https://github.com/boostpro/emacs-llvm-jit)

<comment><USER30> There's no way they'd do that. There's a rift because of LLVM's licensing philosophy: [https://gcc.gnu.org/ml/gcc/2014-01/msg00247.html](https://gcc.gnu.org/ml/gcc/2014-01/msg00247.html)

<comment><USER31> The fact that he even uses the term "adversaries" in this post is disheartening. It's really disappointing that someone who could potentially be such a force for good in the world gets derailed by what is basically tribalism.

Two parties can disagree on 99% of their beliefs, yet still find ways to come together on the 1% that they happen to agree on. These parties can work together to each others' mutual benefit, and the world is better for it. But nooooooooo, not RMS. He has to demonize anyone who is not 100% in lockstep with him on everything, to everyone's detriment, and throw his "adversaries" into a trash bin labelled "Others".

It's a self-fulfilling prophecy. It's easy to drive people apart in the name of some ideal. It takes actual leadership to drive otherwise separate people together to actually accomplish something. I have wished for so many years that RMS would care enough to provide the latter instead of the usual former.

<comment><USER30> > It's easy to drive people apart in the name of some ideal. It takes actual > leadership to drive otherwise separate people together to actually > accomplish something.

Right there is the crux of the matter. RMS is _only_ focused on the ideal. To him, "accomplishing something" is only relevant when the accomplishment is free software that can't be locked away in proprietary codebases.

Viewed through that lens, his actions and beliefs have been remarkably consistent for the past 30 years.

He really grates on people who don't share his idealism, and he's looked up to by people who do. He can be quite polarizing, to say the least.

</comment></comment></comment></comment><comment><USER32> Dream: "Common Lisp/Guile, and a Chromium based rendering engine".

<comment><USER19> Maybe I'm in the minority, but I for one _really_ appreciate how Emacs can run in a terminal, over SSH, with almost no feature-degradation at all.

Besides Emacs being a very nice programming (and programmable editor), this is one the key features which keeps me from even considering other more "modern" options.

<comment><USER33> I haven't used Emacs in a while, so I'm very rusty, but why would you run a remote Emacs in SSH, and not use your local Emacs to access remote stuff via TRAMP?

In my Emacs days, I've never really used the terminal version. Multiple frames, yay!

<comment><USER34> On my remote systems, about the first thing I do after a reboot is call "emacs --daemon" via SSH and then use emacsclient to edit files. For one thing, TRAMP has higher latency, and it also keeps state across disconnects and reboots of my desktop system. I regulary have lots of configuration files open, tail-mode buffers to watch log files, an eshell session, man pages, web pages (using w3m), and it is just too much of a hassle to open these all over again all the time.

Also, if you have to access the remote machine through a low-bandwidth line, the terminal version at least feels more efficient than using TRAMP from my workstation. For most purposes, the GUI does not offer _that_ much of an advantage.

</comment></comment></comment></comment><comment><USER35> Why not start with Hemlock or CLIMacs atop a CLIM implementation, or take the (MIT-licensed) ZWEI codebase and bring that forward? Is there really all that much in GNU emacs that is really _widely_ used?

<comment><USER3> Starting with ZWEI, written in Lisp Machine Lisp in the late 70s, is a very cool way forward. Move back to the 70s and start from there. Good idea. Maybe not.

> Why not start with Hemlock or CLIMacs atop a CLIM implementation,

Why?

</comment></comment><comment><USER36> If we (emacs users) can have an emacs that has a "modern" backend & language but is bug-compatible with the extant elisp code, I think that would be grand.

Particularly nice would be thread support.

I'm pragmatically indifferent as to whether the backend is scheme or common lisp (I'd personally prefer Common Lisp, but whatevs).

</comment><comment><USER37> I spent about 2 years of my life in my 20s hacking on elisp constantly. I was infatuated with it.

Then I spent about 5-10 years using that platform as my IDE.

Guess what... Last year I migrated to IntelliJ IDEA and won't EVER migrate back. It's kind of sad... but IDEA is insanely awesome by comparison.

<comment><USER16> Blasphemy!

On a more serious note, could you elaborate a bit on how IDEA is "insanely awesome" compared to Emacs? What do you find more useful/enjoyable in the IntelliJ IDE?

</comment></comment><comment><USER38> who is Stefan Monnier? and why should we care about what he thinks?

is he the main emacs maintainer? in other words, how seriously should we consider this email?

<comment><USER39> [http://lists.gnu.org/archive/html/emacs- devel/2008-02/msg021...](http://lists.gnu.org/archive/html/emacs- devel/2008-02/msg02140.html)

</comment></comment><comment><USER40> can someone more knowledgeable than me explain how is elisp closer to CL than to Scheme?

<comment><USER3> Both Emacs Lisp and Common Lisp are coming from MIT's Maclisp dialect. There were Emacs variants written in Maclisp or its dialects before GNU Emacs existed. Emacs Lisp was a very simplified Maclisp. Common Lisp modernized Maclisp (especially lexical binding), preserved a lot of Maclisp features and standardized stuff like an Object System (CLOS), error-handling (Conditions), enhanced argument lists (especially keyword arguments).

RMS especially did not like the addition of an object system, lexical binding as the default and keyword arguments. Those three things are quite central in Common Lisp. Emacs Lisp has no object-system in its core, lexical binding has only recently added and it has no keyword arguments.

Scheme is quite different from both. But Guile is a very special Scheme dialect with lots of enhancements, which makes it as large as a Common Lisp implementation. I would expect that code sharing between Scheme implementations is more difficult than between Common Lisp implementations.

<comment><USER41> Keyword argument support is in the Emacs core these days, along with the rest of the CL library.

        (require 'cl-lib)
        (cl-defun foo (&key a b c) (list a b c))
        (foo :c 3 :b 4 :a 1)

</comment></comment></comment><comment><USER42> A lot of people seem to be reading this as if Emacs is choosing between switching languages to Scheme (Guile) or Common Lisp.

Switching to Guile DOES NOT IMPLY switching to Scheme. There is 0 need for compatibility layer or what have you with Guile.

Guile is a language-agnostic virtual machine. It has an implementation for Scheme, but also one for Emacs Lisp. Guile already runs Emacs Lisp faster than Emacs: [https://lists.gnu.org/archive/html/emacs- devel/2010-04/msg00...](https://lists.gnu.org/archive/html/emacs- devel/2010-04/msg00665.html)

The main issue the Emacs developers seem to have with Guile is that it will give developers choices as to whether to write Emacs extensions in Scheme, Elisp, or Javascript/Python/Whatever else Guile supports.

<comment><USER43> >The main issue the Emacs developers seem to have with Guile is that it will give developers choices as to whether to write Emacs extensions in Scheme, Elisp, or Javascript/Python/Whatever else Guile supports

I don't understand why they care? Why would the Emacs developers feel the need to debug compatibility if someone else wrote a shitty extension? The onus would be on the extension developer to fix the extension.

<comment><USER44> Quoting the linked email:

      So if we go for Guile-Emacs, we'll be stuck with Guile, i.e. we'd
      have (old and new) packages that use Elisp, new packages that use
      Scheme, maybe yet other new packages that use, say, Javascript (or some
      other language support by Guile).  That would make the work of Emacs
      (and GNU ELPA) maintenance harder.

<comment><USER45> So don't accept packages into the Emacs core that don't meet their standards? It's okay to have project standards but limiting how other people use your tool seems terribly misguided for a GNU project.

</comment></comment></comment></comment><comment><USER46> Seriously, why not just use Lua? It's perfect for this.

<comment><USER34> I one were to start from scratch, writing an emacs-like text editor by building it on top of a small programming language you can use for customizing and extending the editor, Lua would a great choice. Lua has only a tiny standard library, but in writing the core of the editor, you probably have to supply the equivalent functionality yourself, anyway.

But I do not see how you could bolt Lua on to GNU Emacs. You would have to start from scratch, and that would be a _lot_ of work. And even then, there is the fact, that GNU Emacs is already well-established, has a very loyal user base and is - by and large - good enough, even great for most purposes, so even if you wrote LMacs or whatever one would call it, you would probably end up with, like, a dozen users or so. (IIRC the Zile project, a small emacs-like editor, has tried this, but I do not know how far they got with this.)

</comment></comment><comment><USER47> Why isn't Lua even a choice? It's small, portable, easily embeddable, lightning fast, supports high-level functional programming idioms, and is extremely flexible in creating and enforcing policies (via metaprogramming).

<comment><USER48> Because it is a completely different language and would require rewriting all of the thousands of packages by hand (and centuries of human labor)?

<comment><USER33> OTOH, Emacs isn't that accessible to many users, and its idiosynchrasies can be very offputting. I also quit Emacs after a while because I wasn't totally happy with it.

I think that's why there are so many attempts to built new extensible editors in other languages, like Eclipse, or that Ruby thing there once was (probably still is), or the Chromium/JS monster that was on HN recently.

I haven't yet found an editor I'm really happy with, so if I ever have time, I'll probably write my own as well. I'll lose the Emacs packages I don't use or even know about, but if you haven't bought into an existing ecosystem, that doesn't really matter. I'm unhappy enough with the existing systems that I don't mind throwing them away for something different.</comment></comment></comment>


Nostalgia Just Became a Law of Nature - <AUTHOR> http://nautil.us/issue/21/information/nostalgia-just-became-a-law-of-nature

<comment><USER2> Also check this parallel article (from the comments) [https://www.quantamagazine.org/20140122-a-new-physics- theory...](https://www.quantamagazine.org/20140122-a-new-physics-theory-of- life/)

A bit less romanticized.

</comment><comment><USER3> Stupid popup about something-or-other.

[http://tabcloseddidntread.com/](http://tabcloseddidntread.com/)</comment>


Weather Warning: Earth Could Be Hit by MINI ICE-Age as Sun 'Hibernates' - <AUTHOR> https://www.express.co.uk/news/science/1237178/weather-warning-ice-age-earth-sun-hibernates-solar-minimum-long-range-forecast <comment><USER2> Sources? Stop posting garbage without sources.

<comment><USER3> The whole article is about how it might happen, with only a dozen words strung together at the bottom suggesting it probably won't.

Basically "Prof Valentina Zharkova, from the department of mathematics, physics and electrical engineering at Northumbria University" thinks the sky is falling. Other scientists, who hopefully are from fields that actually study these kinds of things, say it is not.

Like you, I'm siding with the bottom few lines of the article. It's pretty unlikely, and the person saying otherwise isn't even in an authoritative field on the matter.</comment></comment>


It's Tracking Your Every Move and You May Not Even Know - <AUTHOR> https://www.nytimes.com/2011/03/26/business/media/26privacy.html?_r=1&hp

<comment><AUTHOR> Here's the newspaper site if you care to visualize the tracking: <http://www.zeit.de/datenschutz/malte-spitz-vorratsdaten></comment>


What RSA key size to use for an SSL certificate - <AUTHOR> https://certsimple.com/blog/measuring-ssl-rsa-keys <comment><USER2> "NIST says a 2048 bit RSA key has a strength of 112 bits: i.e., there are theoretically 2^112 possibilities to brute force the private key."

This is incorrect. The number of 2048-bit RSA keys is much more than 2^112, bruteforcing is out of question. 2^112 is an approximate number of operations to compute the private key using techniques more efficient than brute force (e.g. number field sieve described later in the article).

<comment><USER3> Author here. That's correct, thanks for spotting the error: indeed, it means using a field sieve as mentioned in the article body, which is certainly not brute force. I've fixed the quote and credited you accordingly.

</comment></comment><comment><USER4> > The results show a 2048 bit RSA key is equivalent to around 116 'bits' of a > symmetric algo. Actually, 116.884, but since, you can't have .884 of a bit > our JS implementation rounds down.

The number of bits is just the log base 2 of some complexity measure. You can certainly have half of a bit in this context.

</comment><comment><AUTHOR> _" 4096 bit handshakes are indeed significantly slower in terms of CPU usage than 2048 bit handshakes."_

I bet they are even slower on mobile devices.

<comment><USER3> Yep, and battery life too. I'll do a followup at some point with latency figures from remote devtools on an android device.

</comment></comment><comment><USER5> Is there a reason 3072-bit or 2560-bit RSA keys can't be used?

<comment><USER6> RSA doesn't restrict key sizes to powers of 2, but powers of 2 are more optimised for implementation.

[http://crypto.stackexchange.com/questions/7849/why-are- rsa-k...](http://crypto.stackexchange.com/questions/7849/why-are-rsa-key- sizes-almost-always-a-power-of-two)

<comment><USER5> the answers you linked to cite key sizes of c * 2^k where c = 1,3,or 5 -- like 3072 = 3 * 1024, 2560 = 5 * 512.

</comment></comment></comment><comment><USER7> The way forward will probably be to switch to ECDSA anyway, just like Cloudflare "Universal SSL".

<comment><USER8> By the time time people switch to ECDSA, they should switch to Ed25519 or Ed448, maybe with ECDSA fallback (with web server and TLS library support for multiple keys for same domain, depending on ClientHello).

<comment><USER9> Have Ed25519 or Ed448 been standardized into TLS? I've never seen a website with that before.

<comment><USER2> They will be: [https://tools.ietf.org/html/draft-ietf-tls- curve25519-01](https://tools.ietf.org/html/draft-ietf-tls-curve25519-01)

<comment><USER8> So far, that's only key exchange, not signatures [1]. CFRG is still debating a signature method [2].

1 - [https://tools.ietf.org/html/draft-irtf-cfrg- curves-07](https://tools.ietf.org/html/draft-irtf-cfrg-curves-07)

2 - [https://www.ietf.org/mail- archive/web/cfrg/current/msg07291....](https://www.ietf.org/mail- archive/web/cfrg/current/msg07291.html)

<comment><USER2> Ah, true. Hopefully, they will use the same curves, with EdDSA.

</comment></comment></comment></comment></comment></comment><comment><USER10> [http://www.wolframalpha.com/input/?i=N[Log2[Exp[%2864%2F9*Lo...](http://www.wolframalpha.com/input/?i=N\[Log2\[Exp\[%2864%2F9*Log\[2^2048\]%29^%281%2F3%29*%28Log\[Log\[2^2048\]\]%29^%282%2F3%29\]\])]

</comment><comment><USER11> That graph of "comparable algorithm strength" would be great if it included the key length of the symmetric cipher.

It was discussed for 2048 bit RSA, but not for 4k or 8k. It seemed like an oversight.

</comment><comment><USER12> One could go for the middle ground with a 3072 bit key.</comment>


Mark Zuckerberg's Angelfire Site (Circa 1999) - <AUTHOR> http://web.archive.org/web/20050529063746/http://www.angelfire.com/ny/mez51/

<comment><USER2> Maybe I'm extrapolating from nothing here, but it's interesting to see that some of the ideas that shaped his career have been with him for a while.

Most notable is his "The Web" applet, which is a sort of extremely elementary prototype of Facebook:

[http://web.archive.org/web/20021201160011/http://www.angelfi...](http://web.archive.org/web/20021201160011/http://www.angelfire.com/ny/mez51/web.html)

Similarly, his remark on his drawing tool page:

 _"The Internet has some effect on almost every person in the world. However, it is fairly uncommon for a person to have an effect on the Internet without going through the long and tedious process of creating a website"_

It seems like from very early on, Mark was interested about how to a) make the web a place for anyone to express themselves, and b) that would allow people to perpetuate their real life relationships.

Again, I may be just reading too much into things here, but I feel that those are interesting thoughts.

I wonder what links people would make between my teenage programming projects and my current work... :)

<comment><USER3> > Most notable is his "The Web" applet, which is a sort of extremely > elementary prototype of Facebook

Right on point.

</comment></comment><comment><USER4> Wow.. "The Web" project seems especially prophetic:

[http://web.archive.org/web/20021201160011/http://www.angelfi...](http://web.archive.org/web/20021201160011/http://www.angelfire.com/ny/mez51/web.html)

 _"As of now, the web is pretty small. Hopefully, it will grow into a larger web. This is one of the few applets that require your participation to work well. If your name is already on The Web because someone else has chosen to be linked to you, then you may choose two additional people to be linked with. Otherwise, if you see someone who you know and would like to be linked with but your name is not already on The Web, then you can contact me and I will link that person to you and put you on The Web. If you do not know anyone on The Web, contact me anyway and I will put you on it. In order for this applet to work, you must E-Mail me your name and the names of the two people that you would like to be linked with. Thank you!"_

<comment><USER5> Another overnight success story.

</comment></comment><comment><USER6> No need for archive.org, his site is still up:

<http://www.angelfire.com/ny/mez51/>

Interestingly, he seems to have gone and re-implemented parseInt and .toString() in JavaScript for his base converter, instead of using the built- in functions: <http://www.angelfire.com/ny/mez51/conv.html>

<comment><AUTHOR> I realized that after I posted the link, then I noticed another link was on the front page, so more or less redundant, but I predict the archive copy will outlive Angelfire.

</comment></comment><comment><USER7> So, he went from Angelfire sites that had big java eyeballs follow your cursor...

to a website where the worlds' governments big eyeballs follow your cursor..

great

<comment><AUTHOR> I'm quite happy that I turned Java off before visiting that site. Then again, it's always off unless I need it.

</comment></comment><comment><USER8> No need for archive.org, it's still up and running: <http://www.angelfire.com/ny/mez51/></comment>


The Truth About Anonymous's Activism - <AUTHOR> http://www.thenation.com/article/190369/truth-about-anonymouss-activism <comment><USER2> There is no new info here, just adding a bashing of Gabriella Coleman (long overdue) on shit we all already know. But Gawker is currently undergoing a long-term assault by 4chan and associated people, by having their scummy behavior and hypocrisy exposed in bite-size infographics and advertisers being emailed and snail-mailed. Because of Adrien Chen's tenure with Gawker (does he have equity?), I'm sure that's what is behind the timing of this article. The article isn't even wrong, but it should be recognized an attrition attack in a war between two Internet cesspits.

Someday I hope for an extensive analysis of Anonymous from someone that isn't Coleman or Chen.

</comment><comment><USER3> An excellent piece, and very thorough. Stories like weev really demonstrate just how far people go to justify horrific ugliness to avoid abandoning the dream.</comment>


A protest of 600k people will result in between 200 and 1100 eventual deaths - <AUTHOR> https://twitter.com/trvrb/status/1269533312759918592 <comment><USER2> This is a very interesting thread with a lot of elements to it. One of the interesting take-aways from it is that the reason this effect is so high is because the US is doing such a poor job of controlling COVID in the first place - so a small increase in infections causes a huge long tail of spreading. The people who get infected at the protest go home and then the disease spreads from them for a long time because the ordinary measures to control the virus aren't very effective. In a country doing a better job of dealing with the virus in the first place this effect would be massively reduced.

<comment><USER3> Could this be addressed by advising people who have attended a protest to self-quarantine for two weeks?

I imagine that if Trump held a press conference with Dr Birx and Dr Fauci, where they advised protestors to stay home for two weeks after attending a protest, he might face some backlash. But at the same time, he gets backlash for "mishandling the Coronavirus." So it's politically untenable for him to give the scientific advice to stay home after protesting. He's damned if he does, damned if he doesn't.

IMO this reveals some hypocrisy on the part of the media pundits painting these narratives, but I digress...

<comment><USER2> Well I don't think we should be making rules for protests. The risks with protests are exactly the same as the risks on public transport or visiting a coffee shop. You risk coming into contact with someone who is infected. That's why the contact tracing is so important. If someone starts suggesting you should isolate simply for attending a rally then the obvious question is should you isolate if you visit a packed casino, or even the Republican National Convention?

</comment></comment></comment><comment><USER4> It's interesting how manifestations about the lockdown were looked down by the majority of the people I've interacted with due to the danger of spreading the disease, but I have not seen most people associate that same risk to the blm protests.

Nor have I seen the egoist label being thrown around. I highly dislike the polarisation people have come to; if I agree with you it's ok, if I disagree you, I question your right to protest and label you an egoist.

<comment><USER5> > It's interesting how manifestations about the lockdown were looked down by > the majority of the people I've interacted with due to the danger of > spreading the disease, but I have not seen most people associate that same > risk to the blm protests.

This sentence is almost incoherent, but if what you're trying to say is that people aren't recognizing that BLM protests might spread coronavirus: here's a prominent Democratic leader associating the BLM protests with spreading the coronavirus:

[https://newyork.cbslocal.com/video/4581115-gov-cuomo- again-u...](https://newyork.cbslocal.com/video/4581115-gov-cuomo-again-urges- protesters-to-get-tested-for-coronavirus/)

<comment><USER3> If you're going to cherry pick opinions, you may as well include the letter from 1,200 health "experts" who effectively called the protests a worthwhile risk.

[https://www.cnn.com/2020/06/05/health/health-care-open- lette...](https://www.cnn.com/2020/06/05/health/health-care-open-letter- protests-coronavirus-trnd/index.html)

<comment><USER5> totony said, "I have not seen most people associate that same risk to the blm protests."

Seems like you just posted 1200 more people who have associated Coronavirus risk with the BLM protests.

You may disagree with people about whether the risk of spreading Coronavirus is worth it, but don't accuse the left of not recognizing that there are risks.

</comment></comment></comment></comment><comment><USER6> Title quibble: coronavirus deaths, specifically, not as a property of protests in general.

<comment><USER7> Also, the estimate is 200-1100 coronavirus deaths per day as a result of the protests, not 200-1100 total deaths.

"...200-1100 deaths per day of protests."

<comment><USER8> I read that as "for each day or protests," not "each day going forward."

</comment></comment></comment><comment><USER9> > but I have not seen most people associate that same risk to the blm > protests.

They know of the risks, but they also know that racism, fascism, police brutality, and subsequent immunity for the latter, are much much much worse to the society than Covid-19, HIV, Ebola and all other combined together, including cancer (1). They don't go away like some virii, they can't be cured like some virii, they don't need physical/blood contact or vicinity to spread, and no lockdown would reduce the risks of "infection". All it needs in their case is corrupt rulers, corrupt press and corrupt law enforcement officers, all enablers that can do damage from whatever distance they wish.

(1- if any of you have a similar serious illness I'm truly sorry about that, but please don't lecture me on the sufferings compared to getting hit by a baton. I have direct detailed accounts from who helps every f'ing day terminal cancer patients and I'm well aware of it, so please try to understand my point instead, thanks)

<comment><USER4> My main point wasn't avout the awareness of the risks (although I have not seen muvh coverage on it).

I understand the risk-analysis, but the people protesting/disagreeing with the lockdown also did the same thing.

There is a sharp contrast in how the blm risk analysis is accepted whereas people were denying that the early protesters had even the right to make that analysis.

</comment></comment><comment><USER10> I think the framing of police brutality and racism as a public health issue is one I've not seen before. It's quite interesting because it gives an apolitical basis for assessing the question of whether the protests are a good idea (albeit one that doesn't take into account all factors, such as discrimination that doesn't lead to physical injury).

</comment><comment><USER11> Its important to take this napkin math with a grain of salt, as even the author admits they could be wildly off with even a small change in transmission rates: "Swapping R0 from 0.95 to 0.9 would decrease estimated downstream infections from 54k-108k to 30k-60k and estimated downstream deaths from 270-1080 to 150-600. Generally, these back-of-the-envelop calculations should be taken with a huge grain of salt. We don't yet have data."

There is also an unstated assumption that none of these people would get infected if they didnt attend the protests, which seems like a big omission.</comment>


How (not) to distribute 100 apples in 10 crates with python.  - <AUTHOR> http://mail.python.org/pipermail/europython/2010-April/007589.html <comment><USER2> It seems like an overengineered solution. A robot arm with image analysis? Why not just stick the apples in a hopper, put in a little door to release one apple at a time, and then just move the hopper back and forth (using a Python script) on a track along the line of crates?

You could also attempt to build a purely physical apparatus sans code to distribute them randomly, but it would take a lot of work to prove it was unbiased. The best solution might be a roulette-wheel style arrangement where the apples are emptied into the middle and move out towards the edges as the wheel spins (you might want to make the crates counter-rotate as you did this). Not bruising the apples might be a challenge.

</comment><comment><USER3> I'm not sure how they're going to stop the pythons from eating the Apples, but if I had to do his homework this is what I'd probably do.

        import random
        boxes = [[] for x in range(10)]
        for apple in range(100):
            random.choice(boxes).append(apple)

        ## don't do it like this
        # [random.choice(boxes).append(apple) for apple in range(100)]

<comment><USER4> why do you not recommend a list comprehension?

<comment><USER5> List comprehensions are wonderful, but in his example it's not being used to generate an actual list (each .append() will return None), just as a strange alternate format for a for loop.

It would be really strange to do

        [random.choice(boxes).append(apple) for apple in range(100)]

when even if you wanted to do it on one line the normal syntax works and would be less likely to confuse:

        for apple in range(100): random.choice(boxes).append(apple)

<comment><USER6> If you're responding to a poorly-disguised lazy homework solution request, then the really strange version is the one you should give. Any student who tries to turn it in will arouse the suspicion of the grader, and they'll deserve it.

<comment><USER3> I had assumed the student would find a decent solution long before this hn thread.

</comment></comment></comment></comment></comment><comment><USER7> This is a pretty awesome response to a not so well disguised homework problem.

</comment><comment><USER6> A simple way to do it is to imagine your apples sitting in a row, assign each one a random number, and sort them by those random numbers. Then the first ten apples go in one crate, the next ten apples go in the next crate, and so on.

But for some reason these guys always seem to want _code,_ not a discussion of how the code might work.

<comment><USER8> That assumes they want an even distribution of apples in each crate. My reading of the problem would suggest that for each apple we should choose a random box and insert the apple into said box.

The question seems like a poorly phrased discussion of the pigeonhole principle.

<comment><USER9> Well, you can extend the parent's solution by throwing in 9 "end of current box" markers with the 100 apples before randomizing the order, rather than declaring them to be every 10 apples after sorting. This is probably not an efficient solution from a programming perspective, but it's similar how you would count the number of distinct such arrangements in combinatorics.

<comment><USER8> As you said, not very practical from a programming perspective, but it's an interesting way of looking at the problem. Thanks.

</comment></comment></comment></comment><comment><USER10> [http://developer.amazonwebservices.com/connect/entry.jspa?ca...](http://developer.amazonwebservices.com/connect/entry.jspa?categoryID=24&externalID=446) may be of service to help ensure true randomness. It may be necessary to create a voting protocol to prevent malicious mechanical turkers from removing entropy from your boxes, and that should also make it suitably self-policing that you will minimize risk of Apple theft. Considering that Apples are among the most expensive consumer electronics, this would be among any poor student's highest concerns.

</comment><comment><USER11> Am I missing the joke here?

<comment><USER12> I don't think the OP was talking about real, physical apples. Paragraph 3 off the response is the payoff, basically.

</comment></comment><comment><USER13>
      M, N = 10, 100
      apples = xrange(N)
      L = random.sample(apples, N)
      crates = [L[i:i+M] for i in xrange(0, N, M)]

<comment><USER14> You are not randomly placing apples in crates. You are putting sequential groups of 10 in crates. 10 is the average number of apples that will be in each crate, but it is possible no crate has 10 apples in it.</comment></comment>


BARF – A Binary Analysis and Reverse Engineering Framework - <AUTHOR> https://github.com/programa-stic/barf-project/?0.2.1 <comment><USER2> Neat! I started with radare2 recently - quite a steep learning curve, but once you understand that it works like VIM (: command etc..) , you are flying ! Worth spending time on learning it. Just make sure you are pulling from git daily as they make several commits/day. Did few CTFs with it (i.e. - trying to learn on tool, but know it well), it can binary patch, detect variables/arguments to functions, and you can rename both args and function names which makes it even easier to read. The only problem is that documentation sucks. You have "cursor mode" so you can essentially move line by line in your asm view, hit return on jump and it moves you to jump location (I like that). It also has ascii-based graph, I do VV (that's the command), it prints the graph, from there you can move between jumps/calls by pressing tab, but even better - if you press t (for true) it moves you to the block that would be followed if jump/call was taken, and f (for false) if it wasn't - also very neat. And that's probably like 1% of what it can do, amazing project.

</comment><comment><USER3> "Not in here mister! This is a Mercedes!"

Can anyone recommend a good tool for exploring and decoding other binary formats? I am interested in analysis of non-code binary data.

<comment><USER4> What binary formats are you interested in? For my graduate thesis I had to read a lot about JPG, PNG, SQLite3 and MS-OLE file formats and I could give you some references to read from and shamelessly link to my github project where you can find some tools related to this (and others) file formats.

<comment><USER5> How would you diagnose an unknown compression format? That's a problem I've encountered recently and I'm hoping there is an easier way than stepping through the reference compressor in a debugger.

<comment><USER6> If you're looking to find out what it is and what's in it, there are some very flexible unpackers out there. Titanium Core from Reversing Labs is excellent.

<comment><USER5> I've already know what it is, and it isn't maleware or a packer. What I'm looking for is something that can do analysis on a bit of data and say what compression scheme is likely used.

<comment><USER6> Ah. Well, the product I mentioned can usually identify packing schemes and unpack them. Commercial close-source, though.

</comment></comment></comment></comment></comment></comment><comment><USER7> Interesting. A few questions:

Is it possible to backtrack using this framework? (I.e., execute code, stop it, go back a bit, continue, etc.)

Is it possible to save/restore the machine state to a file?

And can it deterministically (re)run multi-threaded code? And other (normally non-deterministic) OS functions?

<comment><AUTHOR> > Is it possible to backtrack using this framework? (I.e., execute code, stop > it, go back a bit, continue, etc.)

In the develop branch we have a fully integrated debugger (using python- ptrace), so this should be doable.

> Is it possible to save/restore the machine state to a file?

Nop. If you want to do this, you should try to find a more VM-oriented framework (pyqemu maybe?)

> And can it deterministically (re)run multi-threaded code? And other > (normally non-deterministic) OS functions?

Not yet, but if you instrument some functions using the debugger in the develop branch, you could do it.

edit: PANDAS is also a very good option ([https://github.com/moyix/panda](https://github.com/moyix/panda))

</comment></comment><comment><USER8> I myself reverse engineered two file formats: the Movie Magic Scheduling database files - it's an application-specific project management tool for motion picture and TV production - as well as the Zeni 4 Electronic Design Automation (ie. Electronic CAD) Physical Design documents.

I've been thinking just in the last few days that it would be cool to write up how I did it.

The basic way is to create a document with nothing in it, then a second document with a very small difference. For example in Movie Magic I had one new document that I saved without putting anything in it, then a second where I entered the letter "A" into a certain field.

I made hex dumps of them both, then compared the hex.

Then I would make some modest changes and additions, such as the letter "B" in a different field, or I would change the "A" in that first field to "AB".

This approach only really works if you've had experience implementing file format code; it's particularly helpful to have designed original formats completely from scratch. For these specific reasons I am better than most at file format reverse-engineering, however I don't really have a clue about doing that for network protocols as the Samba folks did.

Once I have some guess about the file format, I start writing a file interpreter, typically to dump the binary format into human-readible text. "Actor's Name: A", "Movie Title: B".

Rather more importantly, that binary file dumper is chock-full of assertions. It's rather more important to get the assertions right than it is to display the actual file payload values.

Loosely speaking, a set of correctly-implemented assertions is, in itself, the documentation for the file format you just reverse-engineered.

It's also important to round-trip. So I come up with some very simple text input format, then a filter that creates documents that are readable by Zeni or Movie Magic, then I try to open the files. If that doesn't puke on my shoes, I edit the file a bit, then run that edited file through my human- readible dumping program.

Lather, rinse and repeat. It is very tedious and slow but it is quite cool when you discover something that works.

I've been puzzling over how I could offer a consulting service where I would reverse-engineer documents as a service to other companies. No doubt there is lots of need for this but I am unclear as to how to market it.

The DMCA only covers Digital Restrictions Management. If the DMCA doesn't apply, as it did not in the case of files that you create yourself as with Movie Magic Scheduling or Zeni 4, while there are some procedures one must take care to follow, reverse-engineering is completely legal.

Another way to put it, is that among the reasons we have patents, is so that reverse-engineering won't be necessary.

<comment><USER9> That tip about assertions is great! It would tell you immediately when one of your assumptions is wrong...

One tip I have, which might be obvious: Write docs as you work. While I was working on an app that reads MS Access file, I simultaneously worked on a guide that documented everything I found out: [http://jabakobob.net/mdb/](http://jabakobob.net/mdb/)

That guide will be useful when I need to fix a bug in the future, and I hope it will also be useful for other people.

<comment><USER8> Indeed. I didn't mention that, but that's something I do myself.

Really the written documentation is far more important than any code. Consider someone who wanted to write a file filter in some other programming language. I use C or C++, they don't look a whole lot like Python or Haskell.

Really the written document should be regarded as authoritative, and not any of the code.

However, it's not clear how to go from a written specification to a working file filter.

One way that would work well but that would be laborious, would be to give the doc to someone who had no prior knowledge of the format in question, so they could write their own filter.

If I gave such a doc to you, then you and I could round-trip our files back and forth between my implementation and yours, then we could be quite confident that the document was correct.

To this very day, OpenOffice will not round trip my resume between .odt and .doc. The are is subtle but I can see it every time I try.</comment></comment></comment>


Ask HN: How to hack a Computer Science minor into a developer position - <AUTHOR>

I will graduate Tufts in December with a minor in Comp Sci (I am a Phil major, but I discovered the joy of problem solving through code late in college, so I couldn't major in it). My focus has been on practical coding courses, not theory (2 in c++, 1 in c, 1 in web dev, and 1 survey of programming languages). I have internship experience that is nontechnical but related to tech: research at a digital ad agency and building a WordPress ecommerce store. I was wondering what HN considers the shortest path from the position I am in to a job as a software developer. (E.g., more school, an independent course, internships, a personal project, working on open source, a nontechnical role at a tech company?) Neither changing location or unpaid work are deal-breakers, I have some savings to stay afloat. <comment><USER2> I was a philosophy major scheduled to graduate in May 2013, so I have some relevant experience. When I left Hamilton in Feb. 2012, I had only 3 classes under my belt and less programming experience than you do now. As an engineer at a large company and now at a start-up, here's what I can tell you.

1) When people ask what I do, I tell them I am a philosopher and a typist[1]. It sounds like a joke, but I'm serious. Most of programming can be broken out between solving the problem and then typing it out (syntax, semantics, style.) For the 'solving the problem' piece, you've been trained to solve abstract problems, to logically work your way towards a holistic, pragmatic solution.

Don't underestimate that.

I reference Wittgenstein and Berlin when writing API documentation. I look to Plato to decouple architecture and encourage sane data-modeling. You have one hell of an advantage. Computer science has been around for 60 years; you're leveraging the wisdom of thousands of years of accumulated problem solving.

2) Do it. You can spend your tuition on learning at a hacker school, but nothing will prepare you like building it yourself. Build yourself a website. It can start by being static with HTML and CSS. Then merge that into a framework (Rails/Django) deployed on Heroku. Then provision your own VPS and deploy it there. You can over-engineer the hell out of a simple website. For a job, never do this. For your personal projects, go apeshit. You need to learn fast. Not all of the implementation details of every technology, but the mental models.

The largest blocker for you right now is that you don't know what to Google. You're smart enough to solve any problem, but you don't yet have the experience to accurately frame them.

I would recommend working as an intern to learn the basics. Get familiar with the problems.

If you have any questions, I'd love to talk more. Shoot me an email at chris at zenpayroll dot com.

(1) [http://twitter.com/comaddox](http://twitter.com/comaddox)

</comment><comment><USER3> As they say on ShopTalkShow, "JUST BUILD WEBSITES!". It took me about 3 months to get to "practical web developer" status with almost no coding background (was previously in marketing). Another 3 months of practice and I was competent enough to get a job doing it.

How did I do it? Total immersion. I spent a few hours every day doing tutorials, and then applying those lessons in little throw-away projects. When not coding or learning, I listened to podcasts in the space.

[http://shoptalkshow.com](http://shoptalkshow.com) by Chris Coyier and Dave Rupert is FANTASTIC, they do a friendly Q&A podcast with guests, sound effects, and it's just a lot of fun overall.

[http://codeschool.com](http://codeschool.com) by the guys at Envy Labs is the best place to learn, bar none. I tried plenty of sources: Udemy, CodeAcademy, Treehouse, and various other platforms. These were all "okay" to "pretty good", but CodeSchool is the best value for your money. The courses alone are worth the $25/mo, but you also get exclusive screencasts that are really great for enhancing your learning.

Stack Overflow is your best friend. I wouldn't recommend doing direct searches on their site when you first run into an issue (just Google your problem) but the vast majority of solutions will come from SO. You can trust the answers on there for the most part, and the best part is that most answers are very informative: they explain the reason for the problem and why the solution works, instead of just giving you some code to copy & paste.

If you're a strong independent learner, you should have no problem following along with the CodeSchool courses (they move pretty fast though, don't feel bad about watching the videos multiple times). Sign up for a month, take a few courses (one or two a week should do at first). Then once you're finished with a lesson, go build a little website (it can be stupid, many of mine were) to demonstrate what you just learned.

If you run into issues, Google it. __This is how you will learn. __Ultimately, no tutorials will take you from 0 to "rockstar ninja guru", that will only come with experimenting. A helpful template for Google searches is [technology] [command or problem]. For example, "move a div with jQuery" or "rails generate migration". Sometimes including the phrase "how to" works, like "how to deploy node app". Also, "is broken", while less specific, can work: "doctype broke my CSS" will probably explain why changing your doctype is causing style anomalies.

Also, just to be clear: I'm not affiliated in any way with any of the websites I mentioned above and have no vested financial interest in them. They're just awesome :)

</comment><comment><USER4> Don't short change yourself, it sounds like you already have some fairly significant programming experience (assuming that's Ming's intro to web dev course and something with Professor Ramsey in there).

> the shortest path from the position I am in to a job as a software developer

Shortest path: apply straight to your first programming job or internship. There are many outfits out there that will take developers with no industry experience who have some practical programming ability.

IMO, the best way to get better is to work alongside the best team you can.

Specifically, if you're interested in web development in Boston:

\- ThoughtBot's apprenticeship [http://www.apprentice.io/](http://www.apprentice.io/)

\- TripAdvisor's internship or web engineering program [http://www.tripadvisor.com/careers/webprogram](http://www.tripadvisor.com/careers/webprogram)

\- Boston Startup Job Fair [http://bostonstartupjobfair.com/](http://bostonstartupjobfair.com/)

\--or just apply to various jobs or internships at companies you know have good programmers.

Worst case, you learn some things during your interviews and get to see the offices of some tech companies. Best case, you get a position and get to learn some stuff.

If you're willing to throw down a few months' time and money, various dev bootcamp programs can practically guarantee you a developer position (and your courses sound like you would have a big head start). There are a couple in Boston, but more established ones in NYC and SF.

If you'd like some help filling in the gaps for interviews or choosing places to apply to shoot me an email: my HN username at gmail. Went through the same process last year.

</comment><comment><USER5> My advice to you would be to improve (or acquire) your software engineering skills through practice, by doing real life projects. What I mean by that is pick an idea/project that excites you (e.g. An interactive website for non- philosophy majors to get a high level overview of philosophy) and do it (I know, "just do it" by itself it not so helpful).

Along the way, you'll probably hit many roadblocks (e.g. ruby? php? html? ftp? git? heroku? aws?) but overcoming them through research, trial and error will teach you a lot about software engineering and you'll come out of the experience more educated. There's also an added benefit of building up a portfolio that shows you care more about solving problems (remember, we started from an idea) than a specific set of technologies (only a tool to execute on said idea).

When you think you're "good enough" to provide value to a company, I would start applying to job openings in parallel. Good luck! I'm actually a mechanical engineer by education, but transitioned to software after graduating using the method described above.

<comment><AUTHOR> Thanks! That makes a lot of sense. It's great to hear that that transition strategy worked.

</comment></comment><comment><USER6> I majored in Biological Anthropology (BA 2007). Today, I'm working as a Software Engineer in San Francisco. I had been exposed to programming in the past, and got my professional start working on consulting projects after I lost my job as a Copywriter in 2009. Work on as many projects as you can and surround yourself with people that know more than you do.

</comment><comment><USER7> Do you have a portfolio, GitHub profile, or some web dev projects that you've built available online? We're always looking for talented junior developers-- feel free to send me a note!</comment>


How many users do you need for a 100MM / year consumer Internet product? - <AUTHOR> http://andrewchenblog.com/2011/02/26/quora-what-is-considered-a-significant-number-of-users-for-a-free-consumer-internet-product/

<comment><USER2> It really depends on the audience as to how much you can sell adverts for (and, of course, on how good you are at selling those adverts).

It isn't at all impossible to get $10-$20 CPM for basic standard banner adverts, then you can charge higher rates for, as an example, video pre-roll adverts, and you can work on bespoke marketing campaigns to complement banner advertising, which again can provide higher profit margins.

The above is from personal experience. For example, the company I've been with for the past few years (essentially since they decided to expand away from being a single website) manages to employ a handful more staff than Reddit on 20m monthly page views.

<comment><USER3> Selling adverts on a quality site is under utilizing a site in many scenarios, IMHO.

If your ad space will be profitable for the advertiser, become your own advertiser. Pivot to generate more direct revenue. I'm doing that on a number of sites of mine and it's working well.

Example: Google is getting into the loan lead generation business ([http://adwords.blogspot.com/2009/10/introducing-adwords- comp...](http://adwords.blogspot.com/2009/10/introducing-adwords-comparison- ads.html)) and the airlines business (<http://www.google.com/press/ita/>) because they know that by letting advertisers get access to their audience, they are leaving a lot of money on the table.

I see their trend towards trying to get more of the pie continuing over time to expand into banking, movie tickets, car rentals and a gazillion other industries until they start coming close to the antitrust lines because it's a huge source of revenue.

</comment></comment><comment><USER4> In stark contrast to B2B where if you get the right niche you can be charging a few thousand per client per month. Even one client == ramen profitable. Then again I guess it's not as cool as having the millons of users required for B2C!

<comment><USER5> One of the things I dislike about B2B is the lack of fine grain control.

With B2C, your revenue grows over time as you grow your userbase. It's pretty predictable, and not too risky.

With B2B, it's more binary. You might go months with 0 revenue, then get some deal and suddenly be profitable. But then there's added risk, as losing a single client can have a massive impact on your bottom line.

<comment><USER6> Sorry and this may get me downvoted, but this comment shows significant naivete about both B2B and B2C business models.

First, neither is easy. But "B2C is predictable and not too risky" is patently absurd. It requires attracting users, keeping them and figuring out a way to monetize them in some economically sensible way. And this is while competing with 1000s of other services trying to attract attention as well. VC portfolios are littered with carcasses of B2C startups that never caught on (and those are the well capitalized ones)

With regards to B2B, I think you're describing B2B in a consulting context vs. B2B SaaS. In SaaS if done right, you have people with a pain and with real money to solve that pain. As a result, you don't require millions of users to make good money.

Of course, one is not better than the other. Depends on what you know, what gets you excited, etc.

<comment><USER5> I've always found B2C advertising supported relatively easy. Perhaps I'm just lucky at it :/ Perhaps I'm just very bad at B2B.

However, I'd much rather have a million customers paying me $1 each, than have 10 customers paying me $100k each.

But I agree with your last point. Depends what gets you excited. For me personally, B2B is as soul destroying as it gets.

</comment></comment></comment></comment><comment><USER7> those numbers are probably off...I mean reddit can be considered a very popular consumer internet product(top 150 on Alexa)...and they just hit 1 billion page views per month, and they are having trouble paying for more than 5 people.

So if $100mm = 100 billion page views/yr. Then by that logic reddit should be making ~12 million a year, which doesn't seem to be that case.

And that would be in addition to the subscriptions people are buying.

<comment><USER8> Reddit is also terrible at business. Half the ads are 'thank you for not using Adblock'. The founders even left it after their contract ended, so I don't think anyone is really looking at the big picture there.

Still, your point is right. You can't just have a blanket number that works for any site.

<comment><USER9> Demographics are also important. Reddit has a much more tech savvy crowd than a site like twitter. I think some of the article's estimates could still hold true for reddit, but it wouldn't be a linear scale. To reach 100B pageviews would require breaking into the mainstream, leading to less sophisticated users and better CPM.

</comment></comment></comment><comment><USER10> Here's a link to the question on Quora:

[http://www.quora.com/What-is-considered-a-significant- number...](http://www.quora.com/What-is-considered-a-significant-number-of- users-for-a-free-consumer-internet-product)

</comment><comment><USER11> I like the sudden advert to Groupon lumped into the Ad-based business model.</comment>