Show HN: Tweeple Search – Find relevant audience and influencers on Twitter - <AUTHOR> https://tweeplesearch.com/ <comment><USER2> I worked on this exact same problem one summer, and it's a much harder problem than I anticipated. The makers of this would probably concur -- I haven't gotten great results, even using their suggested searches (e.g. "entrepreneur brings up Demi Lovato, Tiger Woods, and Pitbull).

There are both tweet-level and account-level signals. But some of those signals can be artificially inflated -- follower counts, favorites, etc. One of the highest signal features I focused on was whether or not you were RT'd or @'d by another influential account. Of course, this is a somewhat-recursive problem which made it difficult to wrangle with.

I eventually got _something_ to work, but it was only possible because I had access to the Twitter firehouse and through the manual weighting of features.

<comment><USER3> @sbuccini This is the exact problem we have solved with our "influencer search" mode. There, we show you influencers based on keywords in _content_ they've shared. :) Not based on their bio text. Though to do, but solves the exact problem you're talking about!

<comment><USER4> This approach is also flawed, as people tend to share different type of content at different times. The same person may be tweeting about a specific conference they attends to, commenting some breaking news or their holidays, etc. They may also change their interest over the course of time.

I created an influencers analytics solution some years ago (tribalytics.com), and Social Network Analysis (SNA) applied to people's followers got us much better results: Instead of analyzing a twitter user's bio, we would also cluster their followers into communities (according to how they follow each other), then summarize each community's characteristics. This gave us multiple accurate segments allowing to qualify a person

e.g.:

* 35% marketing and social media professional

* 10% foodie

* 15% kayak fan

* etc...

I'm not active anymore on this solution though, as that analysis was quite expensive and too complex for what our target customers marketers were looking for...

<comment><USER5> Isn't AleydaSolis involved in that project? Or perhaps it's someone else...

Also who are the big leaders in the influencer search tool space these days?

<comment><USER4> > Isn't AleydaSolis involved in that project? Or perhaps it's someone else...

Indeed, she participated in tribalytics for some time.

> Also who are the big leaders in the influencer search tool space these days?

I'm a bit rusty as the landscape evolves quickly, and I'm not much active anymore on the influence marketing sphere. Two actors which look relevant to me:

* getlittlebird.com

* traackr.com

</comment></comment></comment></comment></comment><comment><USER6> It's an easy to use page, and I enjoyed noodling around for a bit. I didn't get results that I expected, and I'm not sure if that's your product or Twitter. If I had money I might spend a little bit to try the product, but I don't think it's aimed at people like me. Good luck though!

I searched for [suicide prevention] in the UK. I expected to see near the top of the list Professor Louis Appleby (13.6k followers, advises UK government on suicide prevention) and the National Confidential Inquiry into Suicide and Homicide by People with a Mental Illness (only 3.6 k followers, but is the most well known suicide prevention research project in UK)

I got a single page returned, and these two accounts were not on that page. If I change the location from UK to England I get a completely different, worse, set of results, and again these 2 aren't in there.

@ProfLAppleby

@NCISH_UK

(Also, it's weird to see me (300 followers) at number 12 in the UK list.)

Is this something that Twitter sucks at? Is it something to do with lower numbers of people using twitter for suicide prevention tweets?

<comment><USER3> Hi DanBC,

Naren, co-founder of Tweeple Search here. Did you try Influencer Search? Go here: [https://tweeplesearch.com/?bio=suicide%20prevention&is_profi...](https://tweeplesearch.com/?bio=suicide%20prevention&is_profile=true&type=influencers&sort_by=score&page=1&order=desc)

About filtering by location, Twitter allows users to fill free text in the LOCATION field. Hence the disparity. In your example, Professor Louis Appleby has set location as "Stockport station". :)

</comment></comment><comment><USER7> The heuristics seem overly blunt. Clicking on "Entrepreneur", which is the most visible endpoint on the homepage right now, returns a listing of people who have "celebrity" in their twitter profile, with Demi Lovato and her 40m+ follower count at top. Maybe the "influencers" option does something more interesting? I don't know, I accidentally initiated another search by clicking somewhere on the page, then hit the back button a couple of times to land at a page that told me my "daily limit has reached".

<comment><USER3> @danso: Yes, that is the bio search mode. It is simply a bio search sorted by the number of followers. When you switch to influencer search mode, you see results sorted by shares of the articles related to the keyword you searched for. We actually look for keywords in articles they shared by going through their content with NLP.

The free account allows a lot more searches. Do try.

</comment></comment><comment><USER8> I tried something on similar lines,there are some problems with this.Bio's usually don't represent the kind of person or field influencer but instead provide false positives. Eg:- If you search for topics like politics and history, you will find more people who propagate conspiracy theories in the results.Also, sometimes people add their interest in bio's which does not represent their influence in the topic.

</comment><comment><USER9> Or: use the API from twitter (free up to 900 req/15 mins) and sort by followers: [https://dev.twitter.com/rest/reference/get/users/search](https://dev.twitter.com/rest/reference/get/users/search)

What's the added value of Tweeple Search?

<comment><USER3> Hi,

Naren, co-founder of Tweeple Search here. Here are a few points:

1.The Twitter API is not a practical tool you can use to search for tens of thousands of people. Think rate limits. 2\. Results there are not sorted by the number of followers (which is decent marker for influence) 3\. Check out Tweeple Search's filters, they are very powerful 4\. We can give you a list of influencers based on keywords in content they shared as opposed to keywords in their bios (very tough to do this!) 5\. Search is only the beginning. Check out our analysis tools

</comment></comment><comment><USER10> Where do you get the data from? real time via twitter's api?

<comment><USER3> Calling it a trade secret would certainly be a stretch :)

You simply can't do this with Twitter realtime. You'd keep hitting API limits. We use a variety of techniques to collect this data.

Tweeple Search itself is based on the powerful Social Animal content intelligence product : [https://socialanimal.io](https://socialanimal.io)

</comment></comment><comment><USER11> I passed this onto marketing team and they seem to love it. I think a tool like this is what we have to use for more appropriate signups when doing any significant release.

<comment><USER3> Thanks, @navinsylvester!

</comment></comment><comment><USER12> Clicked on vegan and got this as top result:

        pewdiepie @pewdiepie  jackseptic replied to me 12/04
        17/f/Stockholm/Vegan/Lvl6YTHero/

</comment><comment><USER13> Is there any way to do this for Instagram / Facebook / Pinterest?

This seems like it could be super useful.

<comment><USER3> Not really. Their APIs are super-restrictive. Also, their terms of use are another thing altogether.

<comment><USER13> on the twitter front, is there a good way to get secondary influencers that are following a primary influencer and interact with that influencer on a regular basis?

<comment><USER3> You can achieve this for a few accounts with the Twitter API, but due to API limits, it is tough to get it done at scale and make a product out of it. There are creative ways of achieving this, though.

Point in case: take a look at how Nuzzle works for Twitter accounts. It is very close to what you are asking for.

<comment><USER13> thanks for suggesting Nuzzle, I had not heard of this before.</comment></comment></comment></comment></comment>


Part-time founders more productive than full-time founders? - <AUTHOR> <comment><USER2> I built a company as a "part-time founder". I think it's quite do-able and really forces you to focus your time. Parkinson's Law states that tasks tend to expend to fill to time allotted for them... That's true with Founders as well.

That said, I think your day job needs to be fairly light for this to work. At the time, I was doing light consulting after having sold my first company.

I'm currently working on another little startup idea (<http://www.rescuetime.com)> but it's been moving a LOT slower, as my day job is something I invest a lot more time and energy into.

Of course, it very much depends on the scope of the problem you are trying to solve. You're never going to build the next salesforce.com as a PT founder. ;-)

<comment><USER3> The look of your website is very professional. Just curious, did you hire someone? what tools did you use to do it?

</comment></comment><comment><USER4> I'm actually quitting my day job today so I'm betting that a full-time founder is more productive.

</comment><comment><USER5> I wish part time work was easier to find. I would much rather work 20 hours a week for someone else than 40 hours a week. It'd give me a lot more time to work on my own projects.

You can only be so productive anyway. Just because you spend 8 or 9 hours a day at work doesn't mean you're working the entire time you're there. More like 1/3 of the time is actually spent thinking and working. It's not even that you're being lazy, it's just you burn out if you work too hard and too often.

The strange thing is, nobody who's in a position to hire understands this. And if you do find part-time tech work, you're most likely going to be paid LESS per hour than you would if you were doing the same thing full-time.

<comment><USER3> Even stranger: people who work in manual jobs and even in healthcare, do actually work the full 8 or 9 hours. So, maybe we ARE goofing off (me included).

</comment></comment><comment><USER6> short answer: part time sucks. long answer: part time sucks. i did a variant of this (school for part of it, work for another) for 3 years with my first company, and it was frustrating because you miss opportunity after opportunity because you move so slowly, and everything just turns out mediocre.

so if you have to do your startup with a 40-50hr/wk job, do it as briefly as possible until you can prove to yourself the worth of the idea and then make the jump. and this may sound strange, but get yourself in decent physical shape so that you have more stamina and aren't exhausted after a normal 8-9 hour work day. in my case i had way more energy/focus if i was running/lifting regularly (and frittered around/needed more downtime if i wasn't.)

i'm going full time on my idea at the end of this week and have a lot more momentum because 1) other people will be working on it with me and 2) i will be able to focus 100% and do things like pull all nighters when necessary and basically not have any other constraints in the way of getting things done.

</comment><comment><USER7> That only depends. Some people are good only 3 hours a day, but you best believe that they will outdo most programmers who put in 8 hours/day. Nevertheless I would advise you to quit your job and focus full time on your startup.

</comment><comment><AUTHOR> Do you think about your startup at work, and then rush home to implement, knowing you have a limited amount of hours? A lot of startups have begun this way.

<comment><USER8> I had a unique idea that I implemented in my part time. The site was featured on Techcrunch a few years back and quickly started making a lot of money. Within a few months a handful of VC funded competitors sprung up. They were able to iterate on the concept very quickly and completely destroyed me within the year by developing a significantly better product then I did.

The moral of the story: Working on your startup part-time is a great way to mitigate risk while you build out your first version. However, if you see some traction quit you day-job and start executing relentlessly.

<comment><USER9> I think it would also be tricky to explain to your employers why you're working on a startup on the side.

Was your name featured on TechCrunch, or just a URL?

I've tried doing "anonymous" startups under pseudonyms, it's hard to be taken seriously by the likes of TechCrunch, etc.

</comment></comment></comment><comment><USER10> I think Part-time founders are less productive, because they are tired from working for 8 hours. This causes silly mistakes and bugs to happen.

</comment><comment><USER11> full time, definitely. Part time, you are just too tired out.</comment>


Microsoft launching retro t-shirt line - <AUTHOR> http://www.alleyinsider.com/2008/12/microsoft-looks-for-growth-in-hipster-t-shirt-business

<comment><USER2> I think it kind of odd they are celebrating the bill gates arrest, is that something that is considered to be 'cool' in MS culture? Please educate me.

</comment><comment><USER3> Interesting idea, very unique, and even though I use fedora and osx I would rock one.</comment>


7 ways I've almost killed FreshBooks - <AUTHOR> http://entrepreneur.venturebeat.com/2009/09/09/7-ways-i%E2%80%99ve-almost-killed-freshbooks/

<comment><USER2> "Doubt is born out of fatigue and loneliness, and there is a lot of both when you are running a start up. Hang in there and keep your feet moving - there's still a lot of time for you to change the world."

What I needed to hear right now. Right now we're facing very similar competition and finding it hard to crystallise why we are different. Maybe it's time to throw in the towel but sentiments like this keep you going.

<comment><USER3> Don't worry so much about the competition. How many operating systems are there? How may databases? How many bags of chips and loaves of bread and accounting systems and types of yogurt?

Build your stuff to last, keep your customers happy, slowly accumulate more and more and be tight with your money so you don't die.

 _That_ is how real businesses are built and have been built for centuries. Just because we are using "bleeding edge" technology doesn't change anything. How many different technologies have come out in history? Automobiles? Airlines? Television? VCRs, dvds, laser disk?

Focus on your business and your customers and your survival. Most of your customers don't even know your competitors exist and even if there are 10, then odds are if you are all the same, you'll get 10% of the market.

Go to wal-mart and see how many shampoos there are and pain killers.... all those companies are doing fine.

<comment><USER4> Although this message is comforting in my IMO the comparison is not that relevant. There is difference between digital and physical goods when it comes to marginal cost, complexity, and other limits in serving each additional customer.. Also if it is online only marketing with no "human factors" (as in the case of db/OS vendors) involved in the sales process then again the make or break avenues are limited.. Good things are with digital goods you can iterate quickly both in terms of product features and getting the message out.. so if you let nature take its course then a strong and smart competitor will kill you..If you have to win then you have to outsmart at least in a niche..

</comment></comment></comment><comment><USER5> Mike's amazing at 'Marketing 2.0'. Smart dude working in what could have been a dull space that he and the freshbooks crew manage to make interesting.

</comment><comment><USER6> Each point had me saying "ok tell me more!". I wish he had elaborated more on each point.

</comment><comment><USER7> _Thinking we had to move faster than we did...Turns out I was wrong_

This runs contrary to a lot of current wisdom.

I often find myself feeling like I'm moving a little too fast, wanting to say, "Whoa, slow down and think this part of it through a little more." Then I feel guilty. Am I crazy or should I trust my gut?

I wish OP would say a little more about his experience. _Why_ does he feel this way now? A few reference points sure would come in handy.</comment>


Chain World: a game that exists on a unique USB flash drive - <AUTHOR> http://www.wired.com/magazine/2011/07/mf_chainworld/all/1 <comment><USER2> So... to add a little bit more to this story.

I have a copy of Chain World.

I was having lunch with a friend of mine in NYC a few weeks ago, and he was telling me about all this. He said that he was talking with his friend who got the USB drive from Rohrer (I assume this is Ji) and that before Ji sold it, he bootlegged a copy of it. A copy that my friend emailed to me.

I have yet to play (I haven't played Minecraft before, I want a bit of practice before I boot into the game world and die), but it's sitting on my desktop right now.

I think the fact that there is now at least two bootlegged copies going around adds to parallelism to religion. There is "the true path" (the original USB drive), and then there have been two "sects" that have broken off, gone to different parts of the world, and have begun to grow.

I plan on passing my copy from friend to friend on the West Coast. Maybe one day someone will come across copies of two different strains and they'll be able to see the differences that occur after 50 generations of evolution.

<comment><USER3> One question: Do the world save files all end in .dat, or do some end in .mcr?

I ask because some of my code (optimized save file format) was first included in Minecraft 1.3, but from the article it's not apparent which version Chain World is based on.

<comment><USER4> Better, can you get it to be rebroadcast on a server? I'm sure that somebody, somewhere, is gonna be sharing this as soon as they get the world folder off that thing.

<comment><USER2> I wouldn't want to share it like that. Forking the project continues the experience - albeit slightly different than originally intended. If it was uploaded somewhere for anyone to get, it would be no different than any other downloadable Minecraft world.

The cool part isn't the world - it's the legacy of everyone who's played before you.

</comment></comment></comment></comment><comment><USER5> What an absolutely fascinating story. Thank you so much for sharing it with us. As a maker I am always thinking about the rules of what we make, and then once released in the wild we sometimes want to pull our creations back in, and sometimes we can, and other-times we can't. And when we can, those that thought that we were sharing our creations with them, for them to explore and change, feel alienated and rejected, and when we can't those that believed in our original intentions feel betrayed by their trust in us. It's a beautiful and terrifying relationship, and I am glad to be a part of it. Perhaps someday, I'll get to be part of Chain World too.

</comment><comment><USER6> It's interesting that Rohrer played around with the idea of taking Chain World online-- when I heard about Chain World, the first bit of sacrilege that came to my mind was "well, why should this be restricted to a USB stick? There's no reason a multiplayer server couldn't be governed to do this."

I wrote Lonecraft as a way of replicating the Chain World idea, but using an Amazon EC2 instance as a multiplayer server with a few custom Minecraft server plugins and a Heroku instance as a web app to govern the rules of the game (to make sure only one player plays at once and no one plays more than once).

The code is here: <http://www.github.com/ckolderup/lonecraft> It needs some more work to flesh out the idea of resetting the world after a set number of players and exposing the blog entries of each player after the world resets, but the general game mechanic is there and functional.

My instance of it is technically up and running, too, but one thing my friends and I quickly discovered while testing it is that people who have been playing Minecraft for a while have a nasty habit of not... really... dying. Once you've learned the ropes it's pretty easy to make armor, carry food, and fight monsters. Death usually comes only occasionally and as the result of a careless mistake.

I've toyed with the idea of imposing a time limit or a number of logins that you're allowed before you're also kicked off, but that seemed to defeat the whole purpose of giving meaning to the in-game death.

<comment><USER7> Could you make it so that the longer someone plays, the harder it is to stay alive? Armour provides progressively less protection, you require more food, there are more monsters, monsters are more powerful, lightning more frequent etc...

<comment><USER6> It was something I had discussed with some friends who were testing it, but the combination of "would require more work" (although not necessarily too much-- there are various mods based around "survival" concepts that I could theoretically use) plus "gets away from the basic concept of out-of-the-box Minecraft" left me unmotivated to continue.

<comment><USER4> The basic concepts of OOTB Minecraft include bugginess and poor performance. Don't be afraid to make something new.

</comment></comment></comment></comment><comment><USER8> Games are usually not profound and thought provoking for the same reason food is usually not profound and thought provoking. That's just not the path to enjoyment that the medium is naturally inclined toward.

Food, above all else, must be _tasty_. Likewise, games, above all else, must be _fun_ to play -- or more precisely, must provide unusually fulfilling analogs of the plights and labours of real life.

If you manage to make food that is both tasty _and_ cerebrally stimulating, that's pretty awesome. But food that is strictly cerebral is always going to have marginal appeal at best. And you can't accuse chefs of being lazy and unimaginative just because their food isn't cerebral.

(For the nitpickers: yes, food is an extreme analogy. There is more room for expressiveness in games than in food, and it's probably underexploited. But you see the point I am making?)

<comment><USER9> > Food, above all else, must be tasty. Likewise, games, above all else, must > be fun to play

For broad appeal this is true, but says nothing of real value or artistic merit. Expression in every medium is mostly banal schlock, with a few shining examples of quality.

<comment><USER10> I don't see the good in slighting any artist or craftsperson for taking a road less travelled, or in suggesting that an entire medium for creative exploration should constrained by need of easy acceptance.

Art needs to command attention, to force people to think carefully about the ideas that are its payload. It can do this with beauty, or it can do it with ugliness.

To continue the food metaphor: Sea Urchin. Hákarl. Bitter melon. All of which are awful-tasting to most, but delicious to those who have fostered an appreciation for their peculiarities. Anyone who can stomach such foods are offered absolutely unique flavours that defy description or comparison.

It's a powerful tool in the chest of artists, to force their audiences to confront something they may find uncomfortable or unsettling. In every case where the work is truly great, the confrontation is simply an effective mechanism through which something deeply meaningful about the human condition may be revealed. Exemplar are the works of Damien Hurst, Odd Nerdrum, Marcel Duchamp, John Cage, Alvin Lucier, David Lynch, and so many others — all exhaustively-praised once the dust of their highly-confrontational works has settled.

Rohrer's games all exist in this domain, defying casual pursuit, but offering something immensely powerful for anyone willing to overcome their own reservations.

</comment></comment></comment><comment><USER11> If someone wanted to extend this concept a bit further, how about a one-play- only game that could only be passed on as a bequest. That is, no one else gets to play it until you are dead, for real. Bit of a technical challenge, to ensure the game data structures and code could always be run, no matter how much time passed. Also, to use a storage medium with a reliable lifetime of centuries - which Flash isn't.

Some people built an everlasting clock inside a mountain. <http://longnow.org/clock/> It would be interesting to try building a 'game seat' like that - can only be reached by a hazardous journey, will be there and playable for a thousand years, you only get one go at it.

<comment><USER12> > you only get one go at it

Sadly, I don't see how you could hope to enforce this constraint... The idea of a game seat is nice, though -- a collaborative time capsule of sorts.

<comment><USER11> Easy. Any biometric sensor. Fingerprints for instance. If the machine detects the same person trying to play again, it just refuses to run. Or... (if I was designing this) kills them where they sit. Next person that reaches the game's remote location gets to remove the skeleton from the chair. Good incentive to not cheat. (I'm assuming some kind of real-world post-apocalyptic scenario, in which 'legal ramifications' don't exist.)

Actually, if there was some decently functional strong AI in the game, this could get very mythological. Quest to visit the Oracle - you have to get to the site then play the game in order to reach the Oracle in the gameworld. If you die in-game before then - well tough.

</comment></comment></comment><comment><USER13> Rohrer is one of the few game-artist around. I'd encourage everyone to check out his life, career or works of art.

<comment><USER10> There are actually quite a few "game-artists", especially now that the independent gaming scene is thriving. But Rohrer is likely the most interesting for the casual observer. _Passage_ has the startling capacity to reduce people to tears in five minutes flat. His works are absolutely without equal for their inventiveness and resistance to conform to conventional wisdom.

This is my favourite article about him, and one of the most memorable articles I've ever read: [http://www.esquire.com/print-this/future-of-video-game- desig...](http://www.esquire.com/print-this/future-of-video-game- design-1208?page=all)

It's a great starting point for people who are unfamiliar with his story.

</comment></comment><comment><USER14> It kind of sucks that people can't just treat it as is - a game, and go with it, but whatever...a cool idea, nonetheless.

<comment><USER15> The fact it's _not_ "just another game" is part of the point, really.

<comment><USER16> How so?

<comment><USER17> Did you read the article?

<comment><USER16> I did, however, I am not the one making claims that it is "not just another game". I'm pretty sure you have never laid eyes nor hands on it, so I would like to know what you are basing your opinions on.

</comment></comment></comment></comment></comment><comment><USER12> Something along those lines that I'd really like to see (or maybe create someday) would be a decentralized virtual universe. Minecraft and such games are usually run on a server which is under someone's control; it would be funny to have a consensus-based distributed world (drawing inspiration from Freenet and possibly Bitcoin for the technical implementation) which would exist in the network of the players, be the same for everyone, and survive as long as people are playing it.

<comment><USER18> Wow, this is a pretty interesting idea. Have you given any thought on how you'd establish consensus on an ongoing basis? It seems like a hard but very interesting problem.

</comment></comment><comment><USER19> If this were just art, an artist couldn't hope for a better annoying foil than Jia Ji (as described in the article at least).

As it is, I'm just barely skeptical of the whole story -- could just be one of those weird stories. If Rohrer takes the fine art approach to monetizing his games, then his next one will be high concept, and even harder to get, somehow.

</comment><comment><USER20> Not so sure about rule #9. Maybe never play the game in a long time like 9 years may be better. It's like visiting a new country, if the experience is good, I would definitely like to go back and experience the old and the new variations of it.

<comment><USER10> Isn't it obvious? The game is supposed to be a simulation of life. You only live once, and when you die you leave the world forever, with all of your contributions to it remaining for others to build upon.

Without that absolute rule, the game wouldn't have nearly the same symbolic/religious significance, notwithstanding all the meta-game that unfolded hence. Rohrer's son was upset by his father's death specifically because of the significance imparted by this rule.

It makes your time playing the game paramount. You will never play the game again. You will never have another shot at life, so you'd better enjoy it while it lasts.

EDIT: He's an atheist, I'm an atheist. There's an argument to be made for reincarnation, I suppose, but that's hewing close to _respawning_ and it makes me uncomfortable.

</comment></comment><comment><USER21> For a game to be bigger than religion, it must take on a life of it's own that transcends the "lives" of the players.

Chain World sounds very cool, but it's lacking one crucial element, replication. An online version sounds intriguing, but would most likely need substantial tweaks to create the right incentive structure.

</comment><comment><USER22> I posted the same link two days ago but with the original title "Chain World Videogame Was Supposed to be a Religion—Not a Holy War" but it passed completely unnoticed.

Hobbits really are amazing creatures

<comment><USER23> In all honesty, many people don't know what Chain World is (I didn't before reading the article). The sad truth of scrolling through new submissions is that the headlines are the only pieces of information we can rely on. A carefully worded headline can go a long way. Your title (and Wired's title) almost assumes we know what they're talking about -- Wired can get away with that since they're a magazine and this is a featured article that will receive readers regardless (a mysterious title can be beneficial). On HN, it's almost the exact opposite. If people can't tell what you're talking about (in general), it's a lot harder. I guess headlines that are easier to skim are easier to vote for, but I'm waiting for the day that we won't even need to curate content by hand.

<comment><USER22> I completely agree with you. In fact I found the article title from WIRED unsuccessful, but I opted to leave it intact so as to be also easily searchable. It is nice to see that the article receives the attention that it deserves.</comment></comment></comment>


Playing Tetris forever - <AUTHOR> http://tetrisconcept.net/wiki/Playing_forever

<comment><USER2> Disclaimer: I'm a longtime Tetrisconcept.net member and thus a Tetris nerd.

As pointed by Scaevolus, this isn't exploiting the PRNG per se but rather the algorithm that distribute the piece, or in the Tetris jargon "the randomizer" (also "random generator").

The subject of the randomizer is such an interesting one to study.

Very early in the history of Tetris the developers have noticed that the distribution of similar piece was detrimental to the gameplay. Not because necessarily of the famous "get enough S and Z and then you automatically die" effect (that's frankly such an improbable event that you'd better try to do be struck four consecutive time be a lightning bolt), but because it often lead to the creation of a shitty stack.

So they implemented technical measure to correct that. IIRC, Nintendo's version of Tetris for NES and Gameboy were the first to do that (it's a direct detection of repetition: if PieceGenerated == lastPieceGenerated, then reroll for another piece).

Then as time went on, more elaborate system appeared My favorite one is used in the TGM series [0] : it keeps memory of the 4 most recent pieces and denies their distribution if the next generated piece is in that history. There one remarkable thing in its design is that is still distribute the piece if the "rolling of a dice" fails more than 4 times.

By doing that, they managed to have a randomizer that is both reliable (gives enough variety in the distribution of piece so to not be screwed) and unpredictible. It is a very good thing for such a single player focused game.

[0] famous in the internet for the "TGM 3 Tetris Arika !!! Invisible Tetris" and the older "Tetris Japan Finals" videos: <https://www.youtube.com/watch?v=jwC544Z37qo> <https://www.youtube.com/watch?v=mT4TYKJuzx0>

The "7-bag" system that is assumed to be used in this article is a mandatory gameplay element for all modern official Tetris (meaning, as far as I know, you cannot release a game called "Tetris" if you don't use it). As sp332 pointed out, this randomizer uses a metaphoric "bag" throughout which you take all the 7 pieces that it contains. Once you've run out of piece, you put them all back in and you repeat the procedure.

It a simpler system to understand, but it has some flaws.

The main one is that depending how you connect the "seam" of the bags, you can get some unwanted repetitions (e.g finish Bag3 with a T piece and start Bag4 with a T piece as well). This phenomena is even amplified when variant of the 7-bag algorithm is used such as in (now deceased) Tetris Online Japan, where they have a 8 bag strategy or in the New Tetris ("new" here being 1999, and being the N64), where they use a 63 bag (that was actually wanted in order to build 4x4 golden square, which is the gimmick of that version).

One criticism is that it is 100% predictable: you _will_ get any given piece in at least 14 pieces generated. Since the officiel Tetris nowadays seems to relies on multiplayer gameplay (see also: T-Spin and its offensive bonus), having a 100% predictable randomizer could be actually a strength by adding a layer of "fairness".

Further reading: there's an awesome thread [1] in Tetrisconcept.net that talk about randomizer theory. It's a old though, and some image link are dead. Fortunately, archive.org can help us restore them. Two highlights are: A comparison of different randomizer in term of predictability [2] and reverse engineering a randomizer [3] with a graph that gives the probability of having a given piece in exactly X piece [4] or in at least X piece [5].

[1] <http://www.tetrisconcept.net/forum/showthread.html?t=512>

[2] [http://www.tetrisconcept.net/forum/showpost.html?p=20173&...](http://www.tetrisconcept.net/forum/showpost.html?p=20173&postcount=32)

[3] [http://web.archive.org/web/20080326015200/http://www.tetrisc...](http://web.archive.org/web/20080326015200/http://www.tetrisconcept.com/forum/viewtopic.php?p=11682#11682)

[4] [http://web.archive.org/web/20080326015208/http://img249.imag...](http://web.archive.org/web/20080326015208/http://img249.imageshack.us/img249/6804/ncfdhx5.png)

[5] [http://web.archive.org/web/20080326015209/http://img249.imag...](http://web.archive.org/web/20080326015209/http://img249.imageshack.us/img249/6149/tojfdyv0.png)

</comment><comment><USER3> This isn't an exploit of a specific PRNG algorithm, just the constraints specific to Tetris-- 1 of each piece is put into a bag, and pieces are chosen from the bag at random. The bag is refilled when empty.

<comment><USER4> It's also exploiting the highly controversial "hold" move.

<comment><USER5> For what it's worth, a similar pattern could be made without Hold, but it would be far more complicated. The singularly important gameplay element that allows patterns like this is the "bag" style randomizer.

</comment></comment></comment><comment><USER6> A similar concept was used to play a perfect game of asteroids.

[http://www.heise.de/video/artikel/Asteroids-Helmut-Buhler- de...](http://www.heise.de/video/artikel/Asteroids-Helmut-Buhler-der- Praekognitive-1-Platz-1573672.html)

</comment><comment><USER7> _> The following outlines a method of playing forever given the following conditions, which apply in many Tetris products since 2001: _

Ah this explains why things didn't seem right to me. I played hundreds of hours of Tetris as a kid and I would have noticed this statistical pattern, but since my games were pre-2001, I never saw this piece pattern to exploit.

</comment><comment><USER8> Someone used this pattern to make a very simple 'bot' with autohotkey, it is used here: <http://www.youtube.com/watch?v=-aiXCnpXBfc>

That game is event based, not frame based, so it is possible to put down multiple pieces in one frame.

Here is the same bot on a frame based game: <http://www.youtube.com/watch?v=H1y_m1W-go8>

</comment><comment><USER9> If anyone has weeks or months to kill on a totally awesome but utterly useless side project, someone should try to use an emulator to see if it's possible to _actually_ exploit the PRNG on the NES version. I.e., using the fact that you can control the low bits of button press timestamps (or whatever entropy source the game's random number generator uses), see if there's some input pattern (together with cycle-precise input timings to get the PRNG to deterministically do what we want) that creates a fully eliminable set of pieces while also returning the PRNG to its initial state, allowing the sequence to be repeated indefinitely.

You could do this on any version, but the solution would probably be hardware- and implementation-specific; I just mention the NES version because AFAIK it's regarded as the most "authoritative" version.

<comment><USER10> NES might not be a good version for such a project, as most of the time would likely be devoted to working around bugs that start cropping up as you play far beyond the Level 29 "kill-screen." NES was never intended to be humanly playable beyond that point, so weird things start cropping up as you progress that can cause strange behavior or lock up the game.

It could still be possible, but you'd have one more thing to reverse engineer beside just the randomizer. The game does eventually loop back to Level 0 if you get through all the bugs in "the outlands," so I guess as long as you start your game there and are able to manipulate the RNG properly then it may be feasible. The RNG is a LFSR with 32,767 positions, and the selection method is also dependent on the previous piece dealt (starts empty) and number of pieces dealt since power-on (stored in a byte).

(EDIT: There are also ways to indefinitely prolong entry delay via pause cycling. Actually, scratch that, you could even just pause a bit at the end and wait for the proper frame, assuming the other variables are properly aligned. That almost seems a bit unfair though, heh.)

The 1988 Sega arcade version might also be good for such a project. What NES was to North America in terms of being an authoritative "classic" version, Sega Tetris was to Japan. It fills an array with 1000 pieces at the start of each game, so creating some sort of looping pattern would be very possible there.

</comment></comment><comment><USER11> Assuming an infinite series of random tetris blocks, any finite pattern occurs, including one of alternating z- and s- blocks. I don't see how the holding mechanic saves you from this. Anyone care to share some insight on this?

<comment><USER10> The other crucial factor is the 7 Bag randomizer. Pieces are dealt in shuffled sets (or "bags") or seven pieces. The maximum length of alternating S/Z in that case would be four -- xxxxSZ SZxxxx, at the head and tail of adjacent bags.

</comment></comment><comment><USER12> My brain confused Tetris with Pac-Man, the latter of which is impossible to play after level 255, and I worried this was a glitch to get past that barrier, which would've probably meant the death of poor Woz from unending Tetris playing.

<comment><USER13> This page shows how to fix the Pac-Man kill screen: <http://www.donhodges.com/how_high_can_you_get2.htm>

</comment></comment><comment><USER14> i worked on an official tetris game, and so i've had the chance to pour over the official spec. it's been a while, but i believe the official rules for marathon don't allow you to play forever. so, in trying to play forever, you pass up higher scoring moves and then you're cut off at the end of level 19 (again, that's iirc). in multiplayer, trying to play forever is a great way to lose.

the official tetris rules are a thing to behold. i just wanted to throw in my 2 cents worth to keep people from thinking this is an exploit for such an amazingly refined game.

there are modes of tetris that i've played that go to level 20, which means the moment the piece is spawned it's on the floor. but because of that limited maneuverability, i don't think you'd be able to use this strategy very well.

<comment><USER5> Hi guy! I'm the one who made that pattern.

You are correct that it is not optimal play. What it is is a lot more interesting than what it isn't: it's a clear demonstration that survival is trivialized with the "7-bag" random generator. Tetris games used to be enjoyable to play for survival, and that is lost in recent games.

Regarding extreme speeds, you make a good observation. This pattern cannot cope with pieces spawning on the floor. That said, I 100% believe it is possible to construct such a solution.

But the existing pattern stands as something to remind people that Tetris used to be fun without extreme speeds or awareness of scoring gimmicks.

<comment><USER15> Hi colour_thief!

For anyone who doesn't know: I'm not big in the Tetris scene, but I do play a lot of a famous game called TGM2P. colour_thief is pretty famous around the Tetris scene for being one of the few Western Grand Masters around.

colour_thief: I'd love to talk "offline" with you sometime, is there an email I can reach you at?

<comment><USER5> Sent you an email to the address in your profile.

</comment></comment></comment></comment><comment><USER16> Ah, as long as the generator of pieces has a couple properties, this will always work. I find this interesting, mostly because if the generation of pieces is truly random, eventually you will lose; a sufficiently long alternation of S and Z tetrominoes cannot cleanly tessellate.

<comment><USER13> FTA: _Because the random generator provides strings of bags containing each of the 7 pieces in a random order, it is possible to construct a strategy around the relatively small variation, with looping patterns._

There is also a variant of Tetris called Bastet, which always gives you the most unhelpful possible piece :)

<comment><USER17> isn't the most unhelpful piece always just s-z alternating?

<comment><USER18> Not always. When the well consists of mountains and valleys perfect for fitting S/Zs into, Bastet will deal you an O (square).

Though it is possible to get the best of Bastet: its algorithm for determining the worst possible piece works by attempting to drop every piece straight down from all possible horizontal positions, and does not take sliding maneuvers into consideration. Taking advantage of this, you can create a large overhang at the top of the well, then play by building the majority of your stack under it.</comment></comment></comment></comment>


Ask HN: Kubernetes Service Dependencies and Bootstrapping clusters - <AUTHOR> We are using kubernetes. We have about 15 or so pods/services, about 3-4 statefulsets. Things work fine and we are happy with kubernetes so far.
However, for our dev setups and also for CI, we are looking for a better solution. These 15 services have a few hierarchical dependencies. For example, the postgres stateful set should be brought up first, followed by the users service and then the auth service.
The users/auth service will work fine (albeit returning 5xx errors) even without the other services being up. However, if all the services start together, they will all compete for the tiny amounts of CPU that we have given and eventually some starve and kind of gets into weird unrecoverable situations (in CI and test clusters where we can't give max cpu/mem).
I know of ways to increase the cpu/mem limit but keep the initial allocations, small. But I believe that since I know about service dependencies better, if I bring up kubernetes pods/services in a hierarchical way, it would be far more useful for me.
Do any of you that use kubernetes, use any tools/hacks/techniques to bring up pods/services in particular order (without lots of kubectl waits) ? In the past, with `docker-compose` I was able to achieve this via service dependencies but can't find similar in kubernetes. Any pointers ? <comment><USER2> How about some kind of startup condition as a proxy for your original entrypoint?

May I ask why you're running DB and stateful systems in k8s in the first place? The benefits of doing it is not likely to outweigh the difficulty of getting it right.

<comment><AUTHOR> > May I ask why you're running DB and stateful systems in k8s in the first > place?

Because it is the easiest way to get our integration tests in CI working fine. We may change in future, if there is a need.

<comment><USER2> As a suggestion, why not run docker in docker instead? Then you can start your systems serially and finetune network settings, maybe even seed some data etc.

A dockerfile based docker on with simple shell script should do. Maybe even a docker exec but that's pushing it.

I love k8s but in this case it seems like wrong tool for the job.

Tell me uf you'd like some sample code.</comment></comment></comment>


Iterations, a new internal app for managing what we work on next  - <AUTHOR> http://37signals.com/svn/posts/2659-iterations-a-new-internal-app-for-managing-what-we-work-on-next

<comment><USER2> That looks like it could allow extending a practice common in only fairly new startups to more mature and larger organizations. I like the idea of people being able to find things that they want to work on and that they think are important, build teams around themselves and take action. Particularly at the beginning of projects, things work better that way if you've built the right team. Later, you tend to get more pressure from external drivers and end up resorting to "business owners" prioritizing backlogs or whatever applies to your implicit or explicit SDLC or methods.

At first blush, this looks like a way to extend that useful aspects of new teams further by increasing the bandwidth for tracking, communicating priorities and dynamically organizing without losing things. That's how it is different from bug trackers, etc., mentioned in the other comments. It would be interesting to use something like this in a project as mature as 37signals and see what happens.

I will look into it further if it becomes a product.

</comment><comment><USER3> Haven't ticketing systems and bug trackers been around for a long, long, long time?

<comment><USER4> My thought exactly. But I suppose if you limit yourself to 37signals apps, then it was an open need.

<comment><USER5> But isn't this the point of Basecamp?

</comment></comment></comment><comment><USER6> That's funny, I actually use Basecamp for this right now. Specifically with the To-Do lists. I've never really had a problem with keeping track that way, interesting that they wanted something more dedicated to the task.

That being said, if they offered this as a product I'd get on board.</comment>


Sharing made easy - No account required content sharing site - <AUTHOR> http://swithu.com

<comment><AUTHOR> Decided to do a pre release of my newest project <http://swithu.com> Was hoping to have more done before my carpal tunnel surgery (pic on site is mine), but thats as far as I got.

Let me know what u think.

<comment><USER2> Where do you plan to take this?

<comment><AUTHOR> My ultimate goal is [https://docs.google.com/document/d/16uNiMs8a_HaZJtZxHD7jDppI...](https://docs.google.com/document/d/16uNiMs8a_HaZJtZxHD7jDppIS3cuD4jh71lSUoQP1qw/edit)

Basically one site for all types content with revenue sharing for content creators.

(one handed typing sucks)</comment></comment></comment>


Show HN: 1 KB JavaScript library for building applications - <AUTHOR> https://github.com/JorgeBucaran/hyperapp <comment><USER2> Thanks for the great library. I was trying to go-through the source and I have couple of questions.One, on line 135:

[https://github.com/hyperapp/hyperapp/blob/master/src/app.js#...](https://github.com/hyperapp/hyperapp/blob/master/src/app.js#L135)

You seems to be attaching global event handler. Looks like we can't have multiple event handlers at this time. Is this expected or it would be handled in the future?

2\. Why do we use requestAnimationFrame for rendering? Do you think its going to offer best performance all the time? Note here, I'm new to requestAnimationFrame concept, so you can answer me like I'm a noob :)

<comment><USER3> I can answer you #2:

requestAnimationFrame schedules the rendering to when the screen refreshes, which kinda makes sense in this case. (Which is a great thing when you're doing animations or games, since no frames are being dropped) It also doesn't fire while the tab is not active, so that's a plus too.

I've nothing to do with this project though, so I might be wrong on the details.

</comment></comment><comment><USER4> I think it's awesome that you have a minimal piece of library, I hope does get the momentum it deserves (Shameless plug I recently did a hobby project [http://invatar.ga/](http://invatar.ga/) and used moon.js on a second thought I might want to use this instead).

</comment><comment><USER5> This gets reposted every other week. Aggressive marketing team behind this project?

</comment><comment><USER6> Previous HN discussion: [https://news.ycombinator.com/item?id=15264053](https://news.ycombinator.com/item?id=15264053)

</comment><comment><USER7> v0.15.0 is a great release. Feels done. Just needs the router updated for compatibility, and were ready for 1.0 imo.

<comment><AUTHOR> It's being worked right now! :)

</comment></comment><comment><USER8> awesome library. like a breath of fresh, after all those complex frameworks. keep up the good work guys.</comment>


The Most Dangerous Lead - <AUTHOR> https://en.wikipedia.org/wiki/2%E2%80%930_lead_is_the_worst_lead <comment><USER2> I was expecting an article about the most dangerous chemical form of plumbum...

</comment><comment><USER3> Well this certainly holds up in the Belgium vs Japan match today.</comment>


There's more to mathematics than rigour and proofs - <AUTHOR> https://terrytao.wordpress.com/career-advice/there%E2%80%99s-more-to-mathematics-than-rigour-and-proofs

<comment><USER3> _The transition from the first stage to the second is well known to be rather traumatic, with the dreaded "proof-type questions" being the bane of many a maths undergraduate._

I was fortunate that my K-12 math curriculum included proofs and derivations. We learned about sets in first grade. This was a suburban public school using textbooks from a mainstream publisher, in the 70s. Likewise college physics. As a result, there was no transition, and my college math grades went up when the courses were primarily about proofs.

It seems like a couple of things have happened since then, good and bad. What constitutes "math" has been reformulated to give kids a better chance of learning it. In my generation, we did proofs, but it was OK if 90% of the students failed to learn math at all. Math has multiple roles in society, as do science and computer programming, and preparing kids for professional careers in those subjects is not the only reason to teach them.

On the other hand, "math" has also been reformulated to manipulate standardized test scores.

<comment><USER4> I may have inadvertantly down-voted when I meant to up-vote, because mobile phone interfaces are still terrible. HN doesn't let me change the vote. So I want to apologize and say that your comment is spot-on. There's a lot of specifics I'd reply to... if I was at my computer.

<comment><USER5> It sounds like they need a UI change for mobile.

It should be

upvote arrow, username, downvote arrow

instead of the way it is now

Then there would be no accidental downvotes.

<comment><USER4> Yeah, or like on the Stack Exchange sites, where you can undo the actions.

</comment></comment></comment></comment><comment><USER6> I think something similar happens with programmers:

* I vaguely remember ambling around when I first started writing C programs, using a debugger whenever it broke, and generally just slapping for loops and variables together until it worked. I had some intuition for how things worked, but it often took hours of hard debugging. I perceived code purely operationally - "This line sets this variable to 5".

* At some point, you start structuring things into modules, subroutines, etc. You might learn 'best practices' and try intently to practice each one. You'll unit test everything, or otherwise hold strong opinions on how things should be done. Debugging is approached in a more scientific manner. You start to feel like you can get a handle on just about any bug, since you have tools and methods for reasoning about them. There is a lot of focus on verifying code, but not so much on validating. I perceived sections of code as implementing some intuitive concept - "This class represents a bullet, and its methods are reasonable things for working with bullets"

* Now, I rarely spend any time worrying about the best practice of the day just for the sake of it. I generally focus on getting the current task done, but I intuitively know when to apply a certain practice. For example, I might write a test or two after writing algorithmic code. I rarely spend any time at all debugging my own code, since it tends to just work. Since verification is so easy, most of my focus is on validation: "Am I even building the right thing? Will this way of laying things out have acceptable performance characteristics? Does this way of laying out the types mirror how I think about the problem? Is everything I'm writing necessary to solve the problem?" I perceive most code as a formless soup of syntax trees with an operational semantics, and reason about the segments of the trees themselves.

I imagine that Terence's breakdown is relevant to many professions besides just mathematics.

<comment><USER7> For many professional programmers, the rigorous stage of their education never got to the techniques for proving the correctness of programs, or even a realization that such techniques exist. We see this in the discussion of design issues, where informal arguments based on rules-of-thumb, reasonableness, best practices and aphorisms (e.g. "code smells") predominate. There is a tendency for dogmatism to arise in this environment ("thou shalt never write the same expression more than once - always factor it into a function.") (dogmatic thinking is probably confined to a small minority, but because they are vocal, they tend to dominate the discussion.)

I am not one of those who think a transition to formal methods is feasible, let alone that it would solve software's problems. I am also aware that many programmers are effective in putting their general critical thinking and reasoning skills to work in writing correct code (this is one of the reasons why some people without any formal CS education can become excellent programmers.) Nevertheless, and speaking only from my own experience, I think some exposure to more rigorous approaches would help programmers tackle tricky problems, such as those that arise in matters of concurrency or security.

</comment></comment><comment><USER8> My favorite example of mathematics that blends intuition and rigor comes from the works of Archimedes of Syracuse: the _mechanical method_ used to intuit results such as the area of a parabola, followed by the _method of exhaustion_ used to prove the result rigorously.

In the mechanical method, Archimedes imagined, for example, a section of a parabola balancing with a triangle, using the law of the lever (which he also discovered) to derive the area necessary to achieve said balance. He then used inscribed and circumscribed polygons to prove upper and lower bounds on the area thus derived, with (in modern parlance) the two bounds converging in the limit _n_ → ∞, thereby establishing the result. The rigorous method of exhaustion (due originally to Archimedes' predecessor Eudoxus) is effectively equivalent to integral calculus (~2000 years ahead of its time), but guessing the right answer would in many cases have been difficult or impossible without the non-rigorous mechanical method.

Incidentally, the mechanical method itself might have been lost to history had it not been for the discovery of the _Archimedes Palimpsest_ in a medieval prayer book [1], which contains the only known copy of the work describing it. Often called simply _The Method_ , it takes the form of a letter from Archimedes to Eratosthenes, the chief librarian of the Library of Alexandria. When I had occasion to see some original pages of the palimpsest last year (at The Huntington Library in San Marino, adjacent to Pasadena, California), I was struck by the collegial tone of the letter, whose genuine human warmth was instantly recognizable even across two millennia.

[1]: [http://en.wikipedia.org/wiki/Archimedes_Palimpsest](http://en.wikipedia.org/wiki/Archimedes_Palimpsest)

</comment><comment><USER9> The three levels Tao lists (roughly paraphrasable as novice - master - <USER2> master) can be applied to many other endeavors, e.g. art, martial arts, programming, as other point out in other comments. What I'd like to stress is the damage that's generally done in going from Level 1 to 2 in math teaching.

As a specific example take Analysis, generally taught using Rudin. I took this course in my graduate EE studies and _detested_ it. I later thought about this and came to the conclusion that the main reason was the manner of exposition in Rudin where the classical approach is used: each chapter contains an endless sequence of lemmas, minor theorems, etc., one after the other with no discernible purpose and at the end of the chapter you get to prove a big result by using all that machinery. This approach, which dates back to at least Gauss (commonly attributed quote, which I couldn't find the source: "no self-respecting architect leaves the scaffolding in place after completing the building") not only is backward to the real course of events, it sucks the motivation by being so. At least it did so for me and for some other otherwise intelligent friends.

<comment><USER10> It's interesting that you say that. I'm trying to square this with my own experiences - I absolutely loved Rudin and to this day it's one of my favorite textbooks; but at the same time I recognize what you're saying about hiding the scaffolding and I share your distaste towards that.

I guess what I love in Rudin is that he gives the Level 2 details in such a lucid logical manner, with nothing missing and yet relatively tersely, all the details interlocking together. I got a sense of real beauty from reading him as an undergraduate that I did not get from other textbooks. I think that enjoyment ranks much higher from me than the disappointment from not getting the motivation early, which I do try to give myself when I teach something.

I wonder if these two can be separated: if a textbook could be Rudin-style in logical unity, terseness, and beauty and yet not "hide the scaffolding". To some degree, I thought Stephen Abbott's "Understanding Analysis" was a step in that direction, though it was still too wordy and meandering compared to Rudin, for me.

</comment></comment><comment><USER11> I can't believe that none of the people have mentioned Lakatos's "Proofs and Refutations" \- such a good work, which shows the organical refactoring of a vague mathematical intuition into a formal system.

</comment><comment><USER12> There's a lot in this article that reminds me of Bruce Lee's:

"Before I learned the art, a punch was just a punch, and a kick, just a kick. After I learned the art, a punch was no longer a punch, a kick, no longer a kick. Now that I understand the art, a punch is just a punch and a kick is just a kick."

</comment><comment><USER13> I'm curious about HN readers so I made a poll: where you do see yourself in Tao's hierarchy?

[http://goo.gl/forms/KuXqj0wgZu](http://goo.gl/forms/KuXqj0wgZu)

Edit: for those who are interested in the results, here's a one-click link [https://docs.google.com/forms/d/1_RkOUYEoC4m9mYQZKexp2uSuj63...](https://docs.google.com/forms/d/1_RkOUYEoC4m9mYQZKexp2uSuj63wnq24QEGND6xBZ2M/viewanalytics?usp=form_confirm)

<comment><USER3> Post rigorous, heading towards rigor mortis. ;-) This doesn't mean I'm a professional caliber mathematician -- I'm not, and a lot of my math education was in studying physics. It just represents which stages I've passed through, if those stages are valid

It would be interesting to know if there are corresponding hierarchies in subjects such as science, computer programming, or even music.

<comment><USER14> My field is earth science which is, at its best, highly empirical. The stages are a little different: 1) Learning to observe and learning the basics of the theory; 2) Learning the theory quite a bit more and getting more practiced about observing with theory in mind; and 3) Hammering on, and molding or breaking, the theory with more observations or other theory.

Unlike math, it's just really, really difficult to ascertain that the theory is correct and it's never correct at all levels. I have always called geology 'the science of exceptions' because many of our laws are actually tendencies, and given 4.6 billion years, any bug or loophole in the theory will be exploited--and this is when you really learn stuff.

In practice, I actually think the three steps there are kind of fractally embedded throughout in the stages of one's career, one's individual research project, or the overarching evolution of the science.

It's pretty fun.

</comment></comment></comment><comment><USER15> I think I remember hearing the same breakdown of martial arts before.

</comment><comment><USER16> Intuition sounds nice until you realize it means "there are things happening here which I cannot explain nor verify nor teach.".

<comment><USER7> Once you know and respect the difference between intuition and an explanation, it can be very helpful - but you can never trust in it working.

</comment></comment><comment><USER17> this sort of exposition leaves me feeling ill

why base your theoretical structure on satire? ..unless it also is meant to be taken as satire?

why the need to blanket your own experience onto others?

math stands alone, and the curiosity therein also stands alone

i'm arguing myself whether it be systemic or ironic that an academic mathematician would use a limited set to try to categorise mathematical curiosity

why do academics build so many walls and then wonder why they feel so alone?

all you need to be a mathametician is curiosity

</comment><comment><USER18> Mods, Please mark this as 2009.</comment>


Learn hubot the hard way - <AUTHOR> https://github.com/tombell/learnhubotthehardway <comment><USER2> None of the pieces in this repo explain what Hubot is!

Neither does <https://github.com/github/hubot>

The "preface" comes closest; it says Hubot is used for running commands on multiple servers. Does that mean it's like Fabric or Puppet, but invokable from IRC?

<comment><USER3> This book is a work in progress, feel free to open an issue on the repo to suggest things you want to see added, and I'll add them!

<comment><USER4> I was playing around with hubot and especially the redis-brain part of it. Would love to see a good write-up on how to use the brain. I see you're planning to write about it in Chapter 4, so I'd love it when you would add more info about it there!

<comment><USER3> Yup, I plan on writing a pretty detailed section in chapter 4 about the hubot brain. Redis is just the default backend for persisting the brain.

</comment></comment></comment></comment><comment><USER5> To avoid exactly this, I'm working on Instabot: <http://instabot.stochastictechnologies.com/>

It's a hosted bot, similar to hubot, and you'll be able to just go to the site, specify your channel and have Instabot join, without the hassle of setting it up or maintaining it. Of course, it will support all the useful commands, like convenience functions or webhooks.

For early access to the alpha, email me at firstname.lastname@example.com.

<comment><USER3> How does one go about writing their own commands for a hosted product?

<comment><USER5> The commands will trigger a webhook call to your server, where the actual deployment/testing/whatever command you need takes place.

<comment><USER3> I'm trying to understand which part of my book you're trying to 'avoid'. Because from what I understand it's just the deploying bit? Which my hubot- factory already did, and it is being improved for a new version with the help of a great company. You've also essentially made script writing even harder than it is for hubot.

<comment><USER5> I'm trying to avoid people needing to even read the book. It's the difference between rolling your own WebSockets or using Pusher, for example. Some people want things to just work.

<comment><USER3> So you came here to advertise your own product in a post about mine?

You're basically deploying a thing that'll send http requests to another server which then executes specific tasks. So you've essentially replaced hubot with a web application on the end users servers.

What is the difference?

<comment><USER5> The difference is that you don't have to install or maintain it. Standard SaaS.

<comment><USER3> "The commands will trigger a webhook call to your server, where the actual deployment/testing/whatever command you need takes place."

The end user still needs to run something on their servers to actually do anything for their custom commands, which is essentially what hubot scripts do, which is what my book is teaching people about.

<comment><USER5> I know, I'm not disputing anything or devaluing your book. As I said above, some people want things to just work, so we're looking to provide a service for that market. I don't know why you're getting so offended and defensive :/

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> FYI for those interested by a simpler alternative, we have Err our python bot.

<http://gbin.github.com/err/>

Plugins can be install straight from independent public git repos, configured from the chat, with a graphic test console, integrated help system, ...

Writing your own plugins is also a piece of cake <https://github.com/gbin/err/wiki/plugin-dev>

</comment><comment><USER7> Not sure how comparable it is to hubot, but for the last couple of months I've been working on a similar project written in Python called gnotty:

<https://github.com/stephenmcd/gnotty>

Gnotty is primarily a modern web client for IRC, but also features a framework for creating your own IRC bots that can receive and act on data over HTTP.

It's a few days away from its first public release.

</comment><comment><USER8> If you setup Hubot to use Freenode then be aware that you'll want to set:

heroku config:add HUBOT_IRC_UNFLOOD="false"

Otherwise just running /hubot help will flood the channel and it'll get kicked.

<comment><USER3> I recently added a HTTP route to hubot `/hubot/help` that you can visit if you enable the HTTP support to view the help on a page rather than flood channels.</comment></comment>


Alexa Is a Human - <AUTHOR> https://alexaisahuman.com/my-letter-to-amazon <comment><USER2> At [https://www.ssa.gov/oact/babynames/index.html](https://www.ssa.gov/oact/babynames/index.html) you can query the "popularity" (meaning frequency) of a name by year. Alexa seems to have jumped in popularity from 2014 to 2015 and fallen since then, to its lowest level in this century, but not all that dramatically: even in 2018 it was the 90th most popular female name ("0.165 percent of total female births in 2018"; "0.311 percent of total female births in 2015").

Amazon Alexa was announced in November 2014, says Wikipedia, so it makes sense: in 2015 people will have heard the name a lot but not yet understood the implications.

</comment><comment><USER3> I've never wanted a CEO to listen to something more than this.

Maybe someone here can come up with a better, non-human name?

Personally, I think "Slavebot" would be fairly appropriate.

<comment><USER4> "NSA Wiretap"

You can change the name in the Alexia app to anything you want as the keyword to activate it.

<comment><USER3> sure, but that's not the point of the letter. I've just finished reading the whole thing, and it honestly seems to me that those unfortunate enough to have been named Alexa by their parents should have their names legally changed. It seemed funny at first, but now it just seems like a travesty.</comment></comment></comment>


Apple releases ad for Google Glass - <AUTHOR> http://www.apple.com/iphone/videos/#tv-ads-photos-every-day

<comment><AUTHOR> 90% of the shots in this video would be better suited with a Google Glass and not an iPhone. Especially the guy running and trying to take a picture. In my head I screamed, "Don't drop it!" The concert shots, the parents filming their kids, Glass would allow folks to watch in real life HD and have a recording.

I didn't get how popular Glass would be until I saw this ad.

</comment><comment><USER2> GoPro cameras provide a hint about consumer interest in Google Glass. They clearly have a lot of traction and there's a reasonably large market of people who to record their POV while they live their lives. Multiple that demand for POV recording with all of the other smart, heads-up-display abilities of Glass and you can see why people are excited.

There's one thing that this ad does which makes it explicitly NOT an add for Glass - it show people holding the phone. I know people criticize glass for "looking strange." However, it's socially acceptable to hold a camera phone up and take pictures without anyone giving it a second thought. Walk around town with Glass on and people will stare at you. Once a cultural shift happens there won't be a stigma, and then you can see kids, grandmas, and non-techies walking around with them.

</comment><comment><USER3> I've seen this before, but watching it while thinking about Glass was a completely different experience.

It does indeed seem like Glass would have been ideal in most situations covered in the video.</comment>


Elastic Reality 3.1 (1993) - <AUTHOR> https://vetusware.com/download/Elastic%20Reality%203.0%20%2B%203.1%20Update%203.0/?id=10805 <comment><AUTHOR> This software was $3000 per license back in 1993 and was used extensively on TV shows and Movies, most notably the morphs on Star Trek DS9 and the book covers for the Animorphs series.</comment>


Behind the Smiles - <AUTHOR> https://www.revealnews.org/article/behind-the-smiles/ <comment><USER2> What can be done?

You know one should push but either the job as now is left behind jobs of robots or go to chinazi. It is a fight of our era. What is the role of human in job market?

I guess IT we are working is fine by and large. But others?</comment>


Escape From XP - <AUTHOR> http://escapefromxp.azurewebsites.net/

<comment><AUTHOR> Link found on the bottom of [1]. It seems that someone at Microsoft has a good sense of humor :) I'm sure they are also fed up with supporting XP & IE6...

[1]: [http://modern.ie/en-us/virtualization- tools#downloads](http://modern.ie/en-us/virtualization-tools#downloads)

</comment><comment><USER2> Actually quite addictive. :)</comment>


Low cost Cortex-A9 SBC offers GbE and up to 28K FPGA logic cells - <AUTHOR> http://linuxgizmos.com/compact-low-cost-sbc-runs-linux-on-zynq-7007s-armfpga-soc/ <comment><USER2> Be interested to know about I/O between the FPGA and the board. Far easier to prototype if you can shuffle back and forth from memory for those of us that are not EEs.

</comment><comment><USER3> not really low cost at $69 compared to other alternatives, though most if not all alternatives doesn't have FPGA</comment>


How Do You Program a Computer with 10 Terabytes of RAM? - <AUTHOR> http://highscalability.com/blog/2015/8/5/how-do-you-program-a-computer-with-10-terabytes-of-ram.html

<comment><USER2> A 2TB RAM machine with lots of high end Xeon chips is only about $120K, probably less if you are a good negotiator.

I've programmed large RAM machines and it's not that hard. In general, it simply let me run programs I couldn't run previously because they allocated too much memory and crashed or swapped/paged too much.

Having a flat memory hierarchy (all RAM has the same cost) makes it dirt simple. NUMA made it significantly harder because you typically had to structure you program's data, threads to schedule on the appropriate cores or processors.

However, I've found over time that unless you absolutely need to hold all your data in RAM, then spending the additional money to get the largest DIMMs and a motherboard with tons of DIMM slots isn't really cost effective. So long as you can partition your problem, that is a preferred solution in nearly all cases. however, increased programmer productivity can often be more cost effective ("just buy more RAM") and I know people who have been upgrading their computers for years and running the exact same algorithms on their in- memory data sets at very high speeds for years.

Depending on your code, some complex problems can occur. For example, the TLB, which speeds virtual/physical address translations, has its own TLB, and if you're screaming all over memory you can blow out the TLB for the TLB. my experience has been the TLB issues go up with larger memory (since you tend to load more data on each node, and access it with sparser patterns).

I haven't looked at DIMM price curves or roadmaps lately, but I assume that 4TB will be $120K in another 3-5 years, then 8TB will be $120K in 10 years.

With all that said, SSDs greatly reduced the issue of needing lots of memory. I've done jobs that would have required too much physical RAM for my budget by configuring linux swap on a fast SSD. It swapped a lot, but the jobs ran :-)

<comment><USER3> For the programmer who works with far more pedestrian hardware what does someone do with 2TB of ram that you wouldn't implement in lets say a rainbow table on a machine with lesser specs? If you are allowed to say that is.

<comment><USER2> Sure (I'm not sure what rainbow tables have to do with this; those are typically for a very specific use. Bloom filters seem to be a better choice; they allow for fixed-memory hashmaps with known false positive rates).

I had a large multidimensional object and I wanted to smooth it.

So, I loaded the whole object in RAM, did an FFT on it, multiplied it by a gaussian, then did an inverse FFT on it.

Another example would be storing very large hashmaps (as a backing store for a sparse multidimensional array).

If you had a very large biosequence database, and you wanted to map next- generation reads against it, you'd load the database into RAM with an mmap call followed by a block read, then subsequent mappings would be very fast.

Effectively, having large amounts of flat access RAM means you don't have to worry about communication (between nodes, or between node and disk).

</comment></comment></comment><comment><USER4> One interesting aspect is that we've built our computer architecture with the assumption that storage is tiered from expensive to cheap: L1-3 CPU cache, RAM, SSD, HD, maybe Tape.

If just went L1-3 + RAM, it would greatly simplify the job of programmers and programs. (theoretically) no need for virtual memory, buffering / flushing, etc.

</comment><comment><USER5> You start by congratulating yourself for refusing to buy into the Java hype, then continue on with business as usual in C++ or some other non-garbage- collected language.

<comment><USER6> After that you discover it's a NUMA machine, you have a lot of inter-cpu communication and curse you have no GC. Why? Because you bottleneck inter-cpu communication so easily with object synchronization. With GC quite a bit of it can be avoided.

If Java just didn't need to allocate each damn small value object (think struct) separately, I think it'd be a _lot_ better with large heaps.

Array of 1000 non-elementary values could be just one allocation, not 1001. That'd also be 1000 times faster to garbage collect.

<comment><USER7> We're getting this soon(ish)! [http://openjdk.java.net/jeps/169](http://openjdk.java.net/jeps/169)

Still a few years behind other languages, but its coming...

</comment></comment></comment><comment><USER8> Use it to serve PHP pages. That oughta do 'er.

</comment><comment><USER2> Very easily.

</comment><comment><USER9> Using Java?

<comment><USER10> Better use Azul's Zing unless you want worst case heap compactions to take 3 hours (maybe it's better now, per their Pauseless paper, I think, they said 1 second per GiB, but it would have to be a LOT better).

Their approach is to engineer for the worst case, then everything else is easy ^_^. More seriously, their collectors run _all_ the time and GC the whole heap as well as of course nurseries more often.

I'd add that these systems often have NUMA effects in access times, even if it's one big address space, i.e. each chip has better access to the memory directly attached to it than memory on other chips. The mentioned SGI system uses stock Xeon chips, so it's got these sorts of issues I'm pretty sure.

<comment><USER11> Pause times are better than 1 sec/GB with CMS and (by all reports) a lot better with G1. We have a service running an 8GB heap and using CMS and I want to say that it has < 100 ms pauses, definitely < 1s.

With a 64TB heap, the question isn't "is it better?" but "is pause time O(1)?" I can't recall what the Azul collector does, but I think G1 has some kind of linear behavior that would make it unusable on a heap that big.

I don't want to be Pollyannaish, but it is true that as time goes on, people will keep developing technology for handling large heaps. G1 was not fully supported until Java 7 and will probably be the default in Java 9. Meanwhile there's another low pause collector that Red Hat is working on ([http://www.infoworld.com/article/2888189/java/red-hat- shenan...](http://www.infoworld.com/article/2888189/java/red-hat-shenandoah- boosts-java-garbage-collection.html)).</comment></comment></comment>