"Fortnite" saying no to Google's app store could be a great precedent - <AUTHOR> https://www.businessinsider.com/fortnite-epic-games-apple-google-play-app-store-security-2018-8 <comment><USER2> I wonder how Google will react to this: if they truly covet their 30% 'cut' their next releases of Android will adopt Apple-like structures against the practice of sideloading, but if (on the other hand) they covet their relative popularity with Chinese consumers they'll be constrained from doing so.</comment>


Ask HN: Best way to encrypt and password protect dropbox folder on windows - <AUTHOR> Hi All,
We have a company policy meaning I now have to encrypt my work Dropbox on my home PC. So if someone steals my PC they can't access any work files. I'm looking at options for encrypting folders in Window.
I need something that will allow Dropbox to update but require a password to open the folder. Any recommendations? <comment><USER2> Create a partition and encrypt it with BitLocker (built into Windows). Put your local Dropbox there. Pretty straightforward. The only annoying bit is Dropbox will fail to initialize upon windows start when it can't find the folder because the encrypted partition is inaccessible until you unlock it.</comment>


The differences between tinkering and research (2016) - <AUTHOR> http://togelius.blogspot.com/2016/04/the-differences-between-tinkering-and.html <comment><USER4> Thank you OP for your very insightful blog post. You have been better than any other academic at articulating exactly what happens when I (or one of my colleagues) interact with the academic world and you made me realize that the academic world has a set of values that might be good but which I should not even attempt to incorporate in my own professional life.

I have come to my personal conclusion that there is a difference between 'science' and 'the academic world' in much the same way as there is a difference between 'religion' and 'the church'. One can be very religious without going to the church and doing all the rites that are required by the church. In much the same way, a 'tinkerer' (or 'heathen') can be scientific without being academic. I believe that wiki summarizes it quite well for me: "Science is a systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions about the universe". There is no statement about giving proper credit (vanity) or not being allowed to reinvent the wheel without first making damn sure that you are indeed reinventing.

I want to solve real world problems and the way I want to do that is in a structural and repeatable approach. Science helps me doing this. Of course I'm not living in a vacuum, so I will look up as much as I can about the subject at hand as is reasonable within the available amount of time. Interestingly enough, I almost never end up with a research paper, but almost always with blog posts, books, tutorials et cetera. Products don't sell themselves and neither do research papers apparently. I think there is something seriously wrong if that is not perceived as a crisis in the academic world.

<comment><USER5> "a systematic enterprise that builds and organizes knowledge" That is exactly my point. Doing your scholarship is doing your part to organize knowledge. Systematically building knowledge means building on others, and knowing what you build on.

So the quote (which you don't attribute to anyone other than "wiki" \- a citation here would be useful) really proves my point. If you're doing not doing your scholarship, you are not doing science, or research, you are tinkering.

Again, there's nothing wrong with tinkering.

<comment><USER6> Interestingly enough, when I have found research papers on the subject that I am looking at, most are so opaque that it was not worth reading in the first place. More helpful information has been presented by others who you would classify as tinkerers. They have been clearer in their explanations and much easier to build upon.

In the sense of presenting the research in a manner that other people can digest, academia seems to be more of a ancient guild than an organisation for expanding knowledge.

<comment><USER7> Academic papers are written for an audience of experts. If you wish to understand them, you'll need to be an expert in the field (you can sometimes get away with a bit less, depending on the topic). They are not intended for general dissemination.

<comment><USER8> Yes, if you cannot follow review papers you need to read up some more. (Look for review papers if you're new to the area.)

<comment><USER9> Sure, where do I get review papers without having an affiliation with an academic institution?

<comment><USER10> Scihub has made virtually all academic papers freely accessible.

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> I'm an academic researcher. I disagree with most of the article. The main distinction I see between "research" and "tinkering" is not what the author calls "scholarship". From personal experience the only serious distinction is that "researchers" tend to have a very long time attention span compared with "tinkerers".

As for the rest, the goal of academic research, lately, seems to be to produce as many as possible units for the benefit of the publishing industry. In the case of very successful articles, you can fit all the readers in a (small) bus. Most of the articles are badly written, because they follow old habits from the age when printing was expensive. Citations are done in a very selective way. Not so long ago there was no need to give the source of an "unpublished" article, "preprint" was enough. Tinkerers work is almost never cited, even if it is read and used. Blogs? are you kidding me? that's not "scholarship". Heck, even arXiv still seems exotic in some fields, despite the fact that it is a good, standard communication mean for the researchers in other fields. Finally, peer review. How does it work? Researcher B reads what researcher A reports and expresses an opinion, just like any honest reader could.This passes for validation. Magic.

In conclusion I believe the future of academic research comes from "tinkering" with a long time attention span. The dissemination is technically trivial. Peer review will be supplemented by more rigorous validation (although there is no absolute solution here). The same kind of validation will be applied to tinkerers results.

<comment><USER12> _> The main distinction I see between "research" and "tinkering" is not what the author calls "scholarship". From personal experience the only serious distinction is that "researchers" tend to have a very long time attention span compared with "tinkerers"._

Many aspects of "scholarship" become inevitable when "tinkering over a long timeframe". You build up a knowledge base about the problem and/or methodology you're tinkering with, you eventually form a community around your tinkering, etc.

Lots and lots of tinkerers gratify themselves without contributing any modicum of knowledge to the world. But the most successful tinkerers are almost always effectively scholars in the sense of this post. In the sense that they're familiar with a body of knowledge/the people producing that knowledge. And they contribute genuinely new ways of doing things, or new ideas of things to do, or excellent executions on existing ideas that achieve the aspirations of previously articulated ideas. E.g., Linus is certainly a successful OS scholar in the sense of this post.

 _> In the case of very successful articles, you can fit all the readers in a (small) bus_

This is off by at least an order of magnitude for any even remotely reasonable definition of "successful".

 _> Researcher B reads what researcher A reports and expresses an opinion, just like any honest reader could_

First, the author explicitly kinda-sorta agrees with you here ( _...Third, it 's not really the publication..._), so it's pretty clear that publication venue != scholarship.

As an aside, peer review != validation!!! This is something that every young Ph.D. should explicitly learn early on.

<comment><USER11> Yes, I hope that evolution will produce something new from the available academic and tinkerer genes. The academic lives longer in a stable medium, the tinkerer dies young but multiplies faster in chaotic conditions.

</comment></comment></comment><comment><USER13> There are also some unfortunate features of academic research. A big one is that the product the researcher cares about is papers and citations, not working software per se. Papers you only write once. You can cherry-pick examples, work around bugs, do some facile analysis, publish the paper and then move on never to look at the code again.

This means that code and data is typically hacky, unpolished, poorly documented, unmaintained, and unsuitable for public use (or often not publicly inspectable at all). Often the folks doing the grunt work are graduate students, who leave after a few years, leaving a new batch of graduate students to deal with the code without much guidance, so it tends to accrete features but seldom get any refactoring love. Nobody is paying for bug fixes or software maintenance or documentation.

Someone coming from industry (or even just a tinkerer) who wants to use the same idea in a production environment often realizes halfway through reimplementation that it isn't actually better than existing alternatives, and that the analysis/benchmarks in the paper were limited or misleading for a general context. Sorting the wheat from the chaff can be pretty demanding, ideas can sometimes be obfuscated by abstruse mathematical formalism, and academic researchers don't typically go out of their way to make their output usable for non-academics.

<comment><USER14> A few years ago I found myself reading pretty much every paper on methods that could give a better looking solution to "deblurring" an image than the more established methods. This is really a field which has overlap with many different areas of research.

You've got traditional iterative methods through to bayesian methods and machine learning methods. Each of these approaches seemed to be relatively isolated from each other and largely cited from their own subfield, despite all solving variants of the same problem. When they did cite outside the field, they tended to cite less relevant or far from state-of-the-art papers.

What was amazing was the number of papers where you would have a plot showing how amazing the new deconvolution method was in the presence of noise compared to other methods, often using a vaguely defined metric. This plot would take up about half a page, usually following a few example images that were set very very small, often ~2cm and usually in a grid. That was always deeply suspicious.

The other thing I remember were models with tons of unexplained parameters, which were carefully tuned to give good results for that image. But really it was guesswork. Sometimes the authors would carefully avoid well-cited work in another field that they could easily have found.

As a tinkerer with a scarcity of time you learn where not to waste your time and move on.

<comment><USER15> Having tried to find image-processing research/scholarship around 2009, before the neural net explosion really set-in, I have to say the field was a complete rag-bag of fragile, vague methods. And efforts to extend these didn't help them.

Which is to say that no all incremental, scholarly research is necessarily what's needed. In fact, I think what the OP describes is effectively what Thomas Kuhn calls "normal science" [1], which can be great until it becomes terrible - ie, "normal science" periodically reaches crises that require a different approach (inspiration, insight, iconoclasm, etc, whatever).

But that shouldn't detract from the point that anyone should know what traditional research, that if someone is going to engage in something other than normal science, they would benefit from being really aware of why and how they are doing this.

[1] [https://en.wikipedia.org/wiki/Normal_science](https://en.wikipedia.org/wiki/Normal_science)

<comment><USER16> "I have to say the field was a complete rag-bag of fragile, vague methods. And efforts to extend these didn't help them."

You say that like it's not still state-of-the-art in machine learning and AI today. (And piloting self driving vehicles around public roads...)

</comment></comment></comment></comment><comment><USER15> A really excellent summary of the distinctions.

One thing I'd note is that before that Internet era, the scholarship needed to discover who had done X previously took a really significant amount of time. The Internet has changed that a lot, at the same that it has tempted people into tinkering and overall wingnuttery. I remember in 80s, having professor who functioned as little more than a search-engine, tell him an idea and, after a couple of puffs of his pipe, he'd give you a list of people who'd worked on it. He was marvelous despite apparently never having done much more than providing this kind of information.

Another thing to note that scholarship implies a "community of researchers". If you are working problem X and people have looked this problem before, you should think doubly or triply before you claim your step-forward is necessary because people as smart as you already tried to think of everything. The article kind of says this but also consider that the people who have worked on this stuff exist, they have publications, conferences, meetups, etc and you can go to these or look at these and find-out whether or not your idea was considered and rejected, think about what's different between your idea and the idea the "field" rejected and so-forth.

<comment><USER17> > _If you are working problem X and people have looked this problem before, > you should think doubly or triply before you claim your step-forward is > necessary because people as smart as you already tried to think of > everything._

At the same time, such reasoning is used to dismiss or discourage new attempts, often unfairly:

\- each of the earlier people were approaching that problem from a specific angle, and new people might be approaching it from an untried angle

\- the earlier people might have had limited time to really investigate the matter, so there may be unturned stones

\- advances since the earlier work may give newer researchers a leg up.

Note that all of these apply even if (as is likely) the earlier researchers were very smart.

<comment><USER15> >>If you are working problem X and people have looked this problem before, you should think doubly or triply before you claim your step-forward is necessary because people as smart as you already tried to think of everything.

> At the same time, such reasoning is used to dismiss or discourage new > attempts, often unfairly

Well, if you use such an approach of considering previous work, you will have the tools to show you why your work shouldn't be dismissed (whether you are believed or not is a different matter). If you aren't looking at earlier work, not only do people have good ammunition for dismissing your position but it really is hard to be sure you haven't just repeated something.

<comment><USER17> > _if you use such an approach of considering previous work, you will have the > tools to show you why your work shouldn 't be dismissed (whether you are > believed or not is a different matter)_

That doesn't mean you'll be listened to. You might, but that's no guarantee.

There's also a situation you're overlooking: people being discouraged for the cited reason _before_ they even start. "What's the point, if people as smart as you -- and possibly smarter, more senior, and with more of a reputation -- already tried to think of everything?". This can be something that the researcher can tell themselves.

I was explaining why there can still be room for progress, even if lots of smart people have already explored the problem.

> _If you aren 't looking at earlier work_

At no point was I suggesting people shouldn't look at earlier work.

<comment><USER15> Well, as Thomas Kuhn said, science tends to proceed incrementally under normal circumstances. When science reaches a crisis, then there is a tendency to look towards solutions outside the normal paradigms.

Conveniently, computer programming and invention offer other ways for what the OP called "tinkers" to demonstrate the effectiveness of their ideas. Science is imperfect, especially isn't necessarily about the latest, most exciting ideas but about making ideas reliable and the problem with being willing to quickly accept even plausible-sounding ideas is that they can also include a lot of dreck. But hey, there are many alternatives to scientific acceptance.

<comment><USER17> I'm familiar with Kuhn. I'm not saying we should be quick to accept anything, nor am I making any point related to science vs tinkering. All I'm saying is that there are two sides to the statement I was originally responding to, and that they should both be considered.

</comment></comment></comment></comment></comment></comment><comment><USER18> > First of all, it's not about whether you work at a university and have a > PhD. People can do excellent research without a PhD, and certainly not > everything that a PhD-holder does deserves to called research.

I appreciate this dismissed as a straw man, but there are definitely online communities (and individuals I know) who reflexively discount comments from people outside academia.

I understand the instinct, it's a reasonable immune response when everyone thinks they're an expert because they read a blog post or a pop sci best seller.

Sometimes, though, the antibodies kill perfectly healthy tissue too, like valid contributions from bright generalists.

<comment><USER19> another factor is that you are very defensive if you spent a bunch of time getting a phd and someone didnt and is still able to do research as well as you. and im sure its not very common, since getting good at something requires lots of time and energy. but i bet it does sometimes, and thats probably one of the reasons why some of the healthy tissue is killed

<comment><USER20> And they have families to feed and their income depends on thier PhD being enough of a barrier to entry that they can pay thier bills. So much human progress has been lost because the economic model is broken. We seemed to make more progress when science was restricted to a smaller group of practitioners, many of whom were wealthy enough that they could be guided by the science rather than the funding.

<comment><USER21> Also in the early days there was more low-hanging fruit. You could make scientific discoveries by locking yourself in a room for a week and thinking deeply about something you read recently. Then come out and test it with items from your kitchen.

Now it's harder to make breakthroughs without specialized equipment, and the area of interest has narrowed to a tiny sliver.

</comment></comment></comment></comment><comment><USER22> Funny thing: do your eyes light up seeing a "scholarly academic paper", or a "deeply tinkering blog"? Which one have we trained our brains to react more strongly to, what is more exciting to you? What do you feel you learn more out of, have more confidence to choose as a starting point for further activities?

OP's distinction between tinkering and research seems wishful thinking at best.

There are many grievances people have with research (and it's easy to think of examples), but I find most tend to fall under:

* (unaddressed) RISK: poor result analysis, cherry-picked examples; glossing over edge cases and failure modes; focus on novelty and "academic writing" at the cost of repeatability and conceptual clarity. The Mummy Effect of _research that looks great but crumbles to dust when you actually try to touch it_ : [https://rare-technologies.com/mummy-effect-bridging-gap-betw...](https://rare-technologies.com/mummy-effect-bridging-gap-between-academia-industry/)

* (missing) OWNERSHIP: cadence dictated by grant cycles rather than one's intrinsic motivation to solve a problem; slow or missing feedback cycles; no dog-fooding => poor code and documentation; few incentives to make it easier for others to pick up or validate the results

In fact, both of these seem better addressed by tinkerers than by institutionalized research. No wonder people react more positively to "tinkerers" and "hackers"! Academia has a lot of positive inertia (and taxes) going for it, but all goodwill has its limits.

</comment><comment><USER23> As a self-defined tinkerer, I have tried the transition to a peer-review recognised researcher. It is very hard. Science is a very harsh mistress, and what you want your work to be saying is quite distinct from strong tests into what it actually says.

The game-like qualities of surviving peer-review are just that: a set of pretty arbitrary hurdles to cross. If you work in a narrow enough field, your set of blind review peers is tiny and their critique, even if masked, can seem very personal at times. I don't like that much.

I don't overall judge my attempt a success. I think this is an endevour best started early. Late stage career, trying to demonstrate you understand the rules of science, is hard.

<comment><USER24> The best way to get your work accepted in a peer reviewed venue (journal, conference, etc) is to voraciously consume everything that venue has published before, and make your work match the style observed + copiously quote works from that venue.

Obviously the intrinsic quality of your work matters. But you'll more often see work of marginal value that properly follows the form than work of great value that does not follow the form. In fact, in my field (academic HCI), I have never seen the latter, yet see plenty of the former at every conference I attend.

<comment><USER25> Replying here because this matched my train of thought well.

I'm actually really curious to know if anyone has first-hand knowledge of a tinkerer (by this blog's definition) getting their work published.

As the GP said, understanding the rules is one thing, but demonstrating their understanding is another. And as you said, matching the style seems to be more important than the value of work that is demonstrated.

I see the value in matching terms, jargon, style etc just so the reviewers can standardize their thought processes. But as someone who's been a tinkerer for ages now it's hard to change styles for no immediate benefit.

Maybe some examples will inspire me to try :)

<comment><USER23> So my first attempt was 'this is what we do' -a descriptive paper, documenting a technique. The reviews that came back were basically a mixture of 'so what' and 'what does it show' and 'show something of significance' -I had tried to write it as the ur-paper, to establish a technique, and only described in general terms its applicability. They wanted a lot more demonstrable outcomes. I walked away. I found peer review very upsetting. It felt like nobody actually cared about what I was trying to say.

My second attempt was 'this is a specific thing it can do' combined with a much more rigorous academic 'this is the analytical technique' and 'this is a polemic about lack of statistical rigour in results, here is our data, you repeat it' -Which interestingly got panned as 'too much tutorial, too much argument, more results' -which of course this time, we addressed instead of walking away. Result? we didn't get in the first journal/conference, we made the second. I'm reasonably content, but this feels like 'learn the rules of the game' a lot more than 'say something of merit you find personally interesting'

Oh, and 'this technique is interesting' doesn't seem to cut it as a paper subject.

<comment><USER26> Sorry if this comes across as harsh but I'm really not understanding your grievance.

> They wanted a lot more demonstrable outcomes. I walked away. I found peer > review very upsetting. It felt like nobody actually cared about what I was > trying to say.

> Oh, and 'this technique is interesting' doesn't seem to cut it as a paper > subject.

Well, yeah? To invoke an HN cliche, "ideas are cheap". Why _should_ anyone else care about your idea if you can't be bothered to show it actually does something interesting on some specific problem(s) or even motivate why it might be expected to do something interesting in light of what's already out there? Without any expectation of experimental validation, conferences would basically be giant circle-jerks filled with completely inconsequential "interesting ideas".

And it sounds like you took the feedback from your first round of peer review and revised your work in light of those critiques and got your resubmission accepted. That seems like a pretty good experience to me, knowing many academics with multiple experiences of resubmitting work 3+ times (with new results and revisions each iteration) before acceptance. I'm not saying that any peer review process is perfect by any means, but it's a very important filter and in this case it honestly sounds like the criticism you got when your paper rejected was pretty fair...

<comment><USER23> I have no grievance. I gave an experience summary. I think my expectations did not match reality and I was reset.

</comment></comment></comment></comment></comment></comment><comment><USER27> I like this differentiation. Early on in the 3d printer craze I would often say "A lot of people are building 3D printers but only a few people are engineering them." which was much the same argument. When you "engineer" a thing you do so understanding the constraints and design it to reliably within those constraints. When you just build something, as soon as it works once you're done.

<comment><USER28> Engineering has a specific meaning in that area. It means you have a design which can be built using standard techniques and will then work.

<comment><USER27> agreed as long as you also append 'to specifications under all acceptable conditions.'

It was the latter bit that really challenged 3D printer makers. So many 3D printers would work one time and not the next because there were so many parameters that they didn't specifically account for or respond to.

<comment><USER28> Here's one that's very robust.[1]

[1] [https://www.youtube.com/watch?v=srVHzKsBguM](https://www.youtube.com/watch?v=srVHzKsBguM)

</comment></comment></comment></comment><comment><USER29> As someone who spends a fair amount of time with software developed by researchers (mostly biochemistry visualization software), that for all the "research into what has been done", I think there is a huge amount of effort being duplicated all over the place. In my experience this is due to the relatively short timelines of grants, the fact that you have to spend most of your grant money demonstrating that what you created has some kind of scientific impact vs actually implementing the thing, and, as others have pointed out, the fact that a lot of the core work is done by diamond-in-the- rough superstar grad students who will move on after 2-4 years.

</comment><comment><USER30> I wonder if the author would classify the Wright brothers as 'tinkerers'. Overall I find the tone pretty insulting, nobody believes they are the bees- knees quite like academics.

</comment><comment><USER31> Some time ago I wrote about academia vs industry in the context of data science:

In academia, you are allowed to cherry-pick an artificial problem and work on it for 2 years. The result needs to be novel, and you need to research previous and similar solutions. The solution needs to be perfect, even if not on time.

In industry, you should solve a given problem end-to-end. Things need to work, and there is little difference if it is based on an academic paper, usage of an existing library, your own code or an impromptu hack. The solution needs to be on time, even if just good enough and based on shady and poorly understood assumptions.

So, contrary to its name, data science is rarely science. That is, in data science the emphasis is on practical results (like in engineering) - <USER3> proofs, mathematical purity or rigour characteristic to academic science.

(from [http://p.migdal.pl/2016/03/15/data-science-intro-for-math- ph...](http://p.migdal.pl/2016/03/15/data-science-intro-for-math-phys- background.html#priorities))

<comment><USER32> > So, contrary to its name, data science is rarely science. That is, in data > science the emphasis is on practical results (like in engineering) - <USER3> > proofs, mathematical purity or rigour characteristic to academic science.

Science in a nutshell is to take a hypothesis, test it rigorously and validate/invalidate the hypothesis based on the results. The results of this experiment could be extremely practical. Data science that follows the scientific experimental design, does science. The one that doesn't is equivalent to reproduction of past experimental results OR putting a blindfold on your eyes and throwing darts around until it hits the bull's eye.

<comment><USER31> I disagree with this common, yet faulty, explanation. (IMHO coming from people who have never worked in science and for whom everything R&D looks like "science!".)

Building engines or cars (involving a lot of tests!) is considered engineering. And as any serious engineering it's not purely random trails&errors. A/B tests are not a sufficient criterion to make something science. And data science (in business) is focused on solving practical problems (which may or may not be research) rather than open-ended research (which may or may not be practical).

Source: I worked in academia. I now work in data science. You?

<comment><USER32> I have a Ph.D in Computer Science and I have worked across research labs in the industry too so I very much understand where you come from. That said, I don't see the disagreement between us and what part of my explanation you found faulty.

<comment><USER31> Sorry for being combative. It comes from two things:

\- giving talks at universities (then I admit that "data science" is more engineering, compared to academic science)

\- people who had only experience with software development and then any research-like stuff (or simple mathematics) suddenly becomes "science!"

</comment></comment></comment></comment></comment><comment><USER5> I'm the author, and I'm somewhat amused by the negative tone of many of the comments. Seems there are plenty of people with resentments towards research here ;)

<comment><USER33> I think it's because it treats hackers as deficient academics who can't be bothered to cite their sources properly or do any research, while there's an equal and opposite argument that academics are deficient hackers who can't be bothered to put their code on github or write any documentation.

In reality, neither is really deficient, they're just aiming at different audiences which need different things.

<comment><USER5> Oh yes, I'm a terrible hacker, I rarely write code anymore. An unfortunate byproduct of academia is that you get "promoted out of the job", and all the hands-on work is done by your lab members.

I didn't mean to imply that tinkering was inferior to research - the whole premise was just to tease out how they're different, with different audiences, as you say. Interestingly, the discussion here has been dominated by people who think that I look down on them. People who've discussed it in other fora have not read the post that way.

<comment><USER34> Togelius I believe you are in error in Seth Bling's case.

The Show More links under his youtube MarI/O video link directly to the Stanley Miikkulainen NEAT paper.

and he links the wikipedia page on Neuroevolution [https://en.wikipedia.org/wiki/Neuroevolution](https://en.wikipedia.org/wiki/Neuroevolution)

which cites two [1][2] of your papers among others :

yet you say : " _it is atrocious because of the complete lack of scholarship. The guy didn 't even know he was reinventing the wheel, and didn't care to look it up. _ "

a bit harsh perhaps ? At least mentioning the foundational NEAT paper is not a complete lack of scholarship.

I feel you have overlooked that Seth does try and provide good citations to his audience and it would kind if you mentioned him by name in your article rather than _" some guy "_

He did inspire other twitch streamers to experiment with neuroevolution and neural nets and benchmark many SNES games.[3] although downstream this follow up does goes uncited by yourself.

[1] "Neuroevolution in Games: State of the Art and Open Challenges" [https://arxiv.org/pdf/1410.7326v3.pdf](https://arxiv.org/pdf/1410.7326v3.pdf)

[2] "Countering poisonous inputs with memetic neuroevolution" (PDF)[https://www.academia.edu/download/30945872/poison.pdf](https://www.academia.edu/download/30945872/poison.pdf)

[3] Mario Bros mari/o [https://clips.twitch.tv/CourteousEmpathicTruffleCeilingCat](https://clips.twitch.tv/CourteousEmpathicTruffleCeilingCat)

[https://www.youtube.com/watch?v=bRxUQNFxAWc](https://www.youtube.com/watch?v=bRxUQNFxAWc) Mari/o kart winterbunny

[https://clips.twitch.tv/HilariousPolishedZucchiniHassanChop](https://clips.twitch.tv/HilariousPolishedZucchiniHassanChop) \- mario kart RNN mariflow

</comment></comment></comment></comment><comment><USER35> "The difference between screwing around and science is writing it down." \- tpai

</comment><comment><USER36> Whenever i read a paper, i do have troubles understanding the fine details.

That one math formular which is nowhere explained? Yeah good...

I often enough, get the idea and details like results but thats it.

Those papers are not tutorials and they are not written / done to be reproduced and easy understandable. They are, in my opinion, written for other science people who spend there work time doing something similiar or need to solve the problem what the paper solves and rebuild those results in time consuming work.

Probably still better than having no paper but still way more work than just using it.

i do read "the morning paper" ([https://blog.acolyer.org/](https://blog.acolyer.org/)) and he is really good in reevaluating papers. I'm very curious how much time he spends reading and analysing them.

</comment><comment><USER37> > It is easy to miss that someone tackled the same problem as you (or had the > same idea as you) last year

... and hard to resist the temptation to go jump out the window when you find out that the thing that you've been trying to figure out for a year has already been figured out...

<comment><USER13> There are independent benefits to struggling with something for a while by yourself before adopting someone else's solution. For one thing, you will understand the problem and their solution a whole lot better.

<comment><USER38> Pretty much. There are for example studies that show that people who attempt a problem before finding a solution learn a lot more than people who just look at the solution.

</comment></comment></comment><comment><USER39> The word research is pretty vague in comparison to the context the author interprets its meaning. He over-reaches on his explanation of what a researcher is, and leaves too much of a gap between researcher and tinkerer.

<comment><USER40> Maybe the distinction he is trying to make is between academic work and non- academic work (hence the emphasis on scholarship). Non-academics can do research but what makes it academic is its placement in the broader context of civilized endeavors.

</comment></comment><comment><USER41> > Seen as tinkering, that work and video is good work; seen as research, it is > atrocious because of the complete lack of scholarship. The guy didn't even > know he was reinventing the wheel, and didn't care to look it up.

I think its important to note that that he does reference NEAT (and the paper it comes from) and the emulator he used. This is building on what others have already done. While the 2009 paper from the post author would definitely be worth note, I feel "atrocious" is a bit harsh.

</comment><comment><USER42> I tend to disagree with those distinctions and I'm inclined to think the difference is simply that, to a "researcher", doing things is a job; and to a "tinkerer", doing things is a joy.

## Scholarship

To begin with, there are many "tinkerers" out there who practise good scholarship and at acknowledge prior art, as well as provide references. Anyone who is courteous and has the awareness that the world doesn't revolve around her would honestly document these things even if she is a "tinkerer".

And just as there are "tinkerers" who practise good scholarship, there are "researchers" who practise bad scholarship as has been pointed out by @jacobolus.

## Testing

I'm not sure what makes the author thinks that rigorous testing is only a "researcher" quality. The article just insulted all the "tinkerers" out there by saying:

> "Here's another big thing. A tinkerer makes a thing and puts it out there. A > researcher also tests the thing in some way, and writes up what happens."

A "tinkerer" doesn't just make something and puts it out there. A serious "tinkerer" would not want to put something that doesn't work out there—it's embarrassing and it's just, wrong. To make something (reasonably complex) that works usually involves a lot of testing, even if the goal isn't to publish a scientific paper it would still involve a lot of testing.

## Goals

This bit really lost me. It sounds like a bunch of self-contradictory, "researcher"-glorifying statements one after another.

> Usually, goals in research are not just goals, but ambitious goals. The > reason we don't know what the results of a research project will be is that > the project is ambitious; no-one (as far we know) has attempted what we do > before so our best guesses at what will happen are just that: guesses.

A "tinkerer" can have ambitious goals. A tinkerer may do something that nobody has attempted before and, naturally, in that case she would just have to make guesses.

If, hypothetically, I set a goal to investigate the possibility of using CSS to make high-quality 3D games that can run at 60 FPS in web browsers, does that make me a "tinkerer" or a "researcher"? What if, in the process of doing so, I practise good scholarship, document everything carefully in a well- organised format, but just simply have no interest of publishing it in a scientific journal—does that make me a "tinkerer" or a "researcher"?

On this:

> However, if you read a scientific paper those are usually not the stated > reasons for embarking on the research work presented in the paper. Usually, > the work is said to be motivated by some scientific problem (e.g. optimizing > real-value vectors in high-dimensional spaces, identifying faces in a crowd, > generating fun game levels for Super Mario Bros). And that is often the > truth, or at least part of the truth, from a certain angle.

The stated reasons in scientific paper is usually after written __after __the research has been carried out and they are designed to tell a reader why the works is important. In a space where everyone is struggling to keep up with publishing so that they can keep their job, who in the right mind would begin a paper with "our research group embarked on [insert amazing thing] because it seemed fun to do so and has never been done before"? Actually, I would love to read papers like that, because most of the time you know it's the usual standard crap when academics start a paper by stating a problem—it probably means that they haven't solved the problem and are not even close.

Also, I would like to draw your attention to the last, contradictory sentence. It seems that the author isn't quite sure either.

## Persistence

If a someone spends 10 years modifying and perfecting something that has never been attempted before, but does not care for its scientific value, does that make her a "tinkerer" or a "researcher"?

Bottom line, anyone who appreciates, and wants to do, good work will naturally do all of those things above—"tinkerer" or not. To throw away sentences like:

> Probably the most importance difference between tinkering and research is > scholarship.

> A tinkerer makes a thing and puts it out there. A researcher also tests the > thing in some way, and writes up what happens.

> While tinkering can be (and often is) done for the hell of it, research is > meant to have some kind of goal.

> Tinkerers are content to release something and then forget about it. > Researchers carry out sustained efforts over a long time, where individual > experiments and papers are part of the puzzle.

... is just simply conceited.

</comment><comment><USER43> The term research became extremely diluted now. It attempts to put very different things under one roof, and so is not very useful.

Near the end of that blog post you find:

>>Reading the book, I felt that most of my research is not science, barely engineering and absolutely not mathematics. But I still think I do valuable and interesting research, so I set out to explain what I am doing.

Of course, both the blog author and the book author are right in a way. But probably the useful thing to do here would be to come up with new names for these different kind of activities, rather than destroy the language by trying to just place one more thing into "research in general" category.

</comment><comment><USER44> Article author

> The video certainly reached more people on the internet than my work did; it > makes no mention of any previous work.

Video

>"I didn't come up with this on my own, it's based on an algorithm called NEAT based on ap a paper by..."

</comment><comment><USER9> > Scholarship

>Last year, some guy made an experiment with evolving neural networks for Super Mario Bros and made a YouTube video out of it. The video certainly reached more people on the internet than my work did; it makes no mention of any previous work. Seen as tinkering, that work and video is good work; seen as research, it is atrocious because of the complete lack of scholarship. The guy didn't even know he was reinventing the wheel, and didn't care to look it up.

Then publish your damn results in a damn open journal, dammit.

<comment><USER45> Also the video discusses previous work starting at 4:40, so I'm confused what OP is complaining about unless the MarI/O guy re-uploaded with this attribution.

[https://youtu.be/qv6UVOQ0F44?t=4m40s](https://youtu.be/qv6UVOQ0F44?t=4m40s)

<comment><USER46> This is hardly a discussion of previous work. The author of the video states the paper he implemented.

In any semi-decent peer reviewed venue, you would cite a wide variety of papers that solve a similar or related problem or introduce a concept related to the method in the paper.

The related work section of a paper is one of the most important parts since it puts the research into context. By citing other work, the authors explain what has already been done, and what contribution their work makes.

A related work section should also illustrate the downsides, limitations, and differences of other cited research. Limitations of other works are often poorly understood since very few people have had the time to evaluate them beyond the initial experiments done before first publication.

Research is not simply about presenting new techniques but also understanding the trade-offs that arise when choosing different solutions to a problem.

<comment><USER9> Replicating the results of existing papers is important too

<comment><USER47> That is a different issue, though. The "Related Work" section is there to tell the reader about related approaches to similar problems, previous research on the same topic, etc. It's where you distinguish your work from what was previously done, and justify why this is a meaningful distinction. It's "what others have done and why we don't just use that".

Reproducibility is important but a completely different topic and I don't know why you're bringing it up here.

</comment></comment></comment></comment></comment><comment><USER34> OP, Togelius, is quite wrong in 1 point at least.

tl;dr Seth Bling directly references NEAT and his links[Show More [1]] link to 2 Togelius papers[13][14] among many others !

To wit, in OP, Togelius mentions Seth Bling's MarI/O[1] as lacking scholarship because he didn't mention Togelius:

" _seen as research, it is atrocious because of the complete lack of scholarship. The guy didn 't even know he was reinventing the wheel, and didn't care to look it up._ "

Rather disdaignfully (sadly/ ironically/ poigniantly) Togelius refers to Seth not by name but only as " _some guy_ ".

Yet Seth Bling[2] in his short video, mentions and explains NEAT and directly links Ken Stanley's and Risto Miikkulainen's foundational NEAT paper[10] at U.texas which Togelius builds on [hint: click SHOW MORE under Seth's video]

And much more, i.e.: ---- the innovative toolchain Seth uses: Bizhawk to script SNES emulators in Lua [12]!

\---- Seth links the wikipedia page on Neuroevolution[11] which directly references 2 of Togelius's own papers[13][14]

Seth's [Show More] Links on the MarI/O video[1] are very very comprehensive and (IMHO) indeed a great example of scholarship !

Togelius is a top flight scholar, and publishes in the field of AI game players and level generators[15] so I hope he will welcome some random _tinkerer_ pointing out his error.

[full disclosure: I am a Togelius fan]

To give Togelius his due he does at least link to Seth's MarI/O video.

Perhaps Togelius failed to click / read the Show More links ? The charitable benefit of doubt principle is accidental oversight.

Seth Bling is primarily a Twitch Streamer[2] and Speed Runner[3] who often popularises complex programming[4], math and academic ideas[5] to a primarily young audience using Mario and Minecraft[6] and takes requests via his subreddit[7]

Seth's very popular Youtube video MarI/O which uses NEAT (Neuro Evolution of Augmenting Topologies] to learn and play Super Mario World, has led to a huge number of streamers on twitch playing other MarI/O levels[16], using RNNs, LSTMs, and other SMW levels and SNES games, like RNN multiplayer x4 Mario Karts[17].

The OP article is great, academic rigor and bibliographic scholarship might _sometimes_ differentiate the full time university scholar from the amateur researcher, enthusiast or tinkerer.

Seth and his many followers may not be 'scholars' as in academic research but it is an amateur community of mostly youngsters doing their own research by having fun and sharing ideas, mostly undocumented outside their own transient twitch streams.

Compare this to the well respected amateur astronomy community and perhaps a responsible academic approach is to try to reach out and tap in rather than adding to the distance.

By his own OP standards of Togelius is lacking in that hasn't discovered/ mentioned the work done by the Twitch/ Bizhawk community - it is not entirely undocumented [16].

Togelius your AI Mario competition[8] is legendary in academic AI circles but not necessarily widely known by youngsters playing Mario ( and to be fair AFAIK you don't publish in their forums like twitch) although the code Infinite Mario[9] you base your work on is by Marcus Persson[10].

Though Persson is better known as Notch [10] ( author of Minecraft ) who is more famous than perhaps even Alan Turing , Yann le Cun, Jurgen Schmidhuber, or Geoff Hinton to the under 16 crowd.

Perhaps Togelius could mashup some of his work and do a crossover episode with Seth or relaunch the Infinite Mario competition[8] / benchmarks to this new younger crowd evolving Neural Topologies to play Mario games - or even help document some of their fun.

As public servants funded by tax dollars it is perhaps the responsibility of the academic to reach out rather than blaming fun loving amateurs popularising and doing research / 'tinkering' for not putting in the due diligence of scrupulously mentioning every source.

[IMHO] I agree amateurs don't always do full and proper scholarship and links are food for hungry minds - but they aren't publicly funded researchers but amateurs working for love of the subject in their spare time, for free, or for streaming donations and youtube hits, using transitory, live media and whose audience must be constantly engaged and may have shorter attention spans than the average academic.

[full disclosure I am an amatuer neural net nerd]

[1]Seth Bling's MarI/O [https://www.youtube.com/watch?v=qv6UVOQ0F44](https://www.youtube.com/watch?v=qv6UVOQ0F44)

[2][https://www.twitch.tv/sethbling](https://www.twitch.tv/sethbling)

[3][https://www.youtube.com/watch?v=-spFoon7klA](https://www.youtube.com/watch?v=-spFoon7klA) sub 1 min SMW via credits warp

[4]Bling's arranging shells in SMW to human inject Flappy Bird code into SNES [https://www.twitch.tv/videos/57032858](https://www.twitch.tv/videos/57032858)

[4]Bling's Atari 2600 emulator built in Minecraft [https://www.youtube.com/watch?v=5nViIUfDMJg](https://www.youtube.com/watch?v=5nViIUfDMJg)

[5]Seth Bling's RNN Mario Cart race [https://www.youtube.com/watch?v=Ipi40cb_RsI](https://www.youtube.com/watch?v=Ipi40cb_RsI)

[6]Bling's Minecraft Redstone tutorials [https://www.youtube.com/watch?v=DzSpuMDtyUU&list=PL2Qvl4gaBg...](https://www.youtube.com/watch?v=DzSpuMDtyUU&list=PL2Qvl4gaBge1kABr3aBCrMSb94HFLIjbn)

[7][https://sethbling.reddit.com](https://sethbling.reddit.com)

[8]Togelius 2009 Infinite Mario AI competition [http://julian.togelius.com/mariocompetition2009/](http://julian.togelius.com/mariocompetition2009/)

[9][https://web.archive.org/web/20080423023424/http://www.mojang...](https://web.archive.org/web/20080423023424/http://www.mojang.com/notch/mario/)

[10]Stanley Miikkulainen NEAT paper [http://nn.cs.utexas.edu/?stanley:ec02](http://nn.cs.utexas.edu/?stanley:ec02)

[11] [https://en.wikipedia.org/wiki/Neuroevolution](https://en.wikipedia.org/wiki/Neuroevolution)

[12] Scripting Lua in Bizhawk Emulator [http://tasvideos.org/Bizhawk/LuaFunctions.html](http://tasvideos.org/Bizhawk/LuaFunctions.html)

[13] Risi, Sebastian; Togelius, Julian (2017). "Neuroevolution in Games: State of the Art and Open Challenges" (PDF). IEEE Transactions on Computational Intelligence and AI in Games.[https://arxiv.org/pdf/1410.7326v3.pdf](https://arxiv.org/pdf/1410.7326v3.pdf)

[14] Togelius, Julian; Schaul, Tom; Schmidhuber, Jurgen; Gomez, Faustino (2008), "Countering poisonous inputs with memetic neuroevolution" (PDF), Parallel Problem Solving from Nature [https://www.academia.edu/download/30945872/poison.pdf](https://www.academia.edu/download/30945872/poison.pdf)

[15] Togelius' Game AI book [http://gameaibook.org/](http://gameaibook.org/)

[16] [https://clips.twitch.tv/CourteousEmpathicTruffleCeilingCat](https://clips.twitch.tv/CourteousEmpathicTruffleCeilingCat)

[16] [https://thenextweb.com/artificial- intelligence/2018/01/03/th...](https://thenextweb.com/artificial- intelligence/2018/01/03/this-live-stream-of-ai-learning-to-play-super-mario- bros-is-awesome/)

[17] [https://www.youtube.com/watch?v=S9Y_I9vY8Qw](https://www.youtube.com/watch?v=S9Y_I9vY8Qw)

<comment><USER48> Thanks for the commentary togelius [1, 2] and deepnet [above].

Anyone interested in adding a soupçon of scholarship to Seth's project?

Words do mean different things to different people, in different contexts..

There is a recent body of literature that explores the modern "maker" [3] movement. However, "maker" as a term may not have been a good fit for the OP's argument [1], which contrasted (academic) researchers with so-called "tinkerers".

An alternative term for "tinkerer" might be "bricoleur", a loanword from French. (Roughly, it still means one who tinkers: [https://en.wikipedia.org/wiki/Bricolage](https://en.wikipedia.org/wiki/Bricolage) but has other meanings depending on the academic lens.)

Given that we are discussing AIs that play, in the context of education, we can also go back to Seymour Papert's work on [https://en.wikipedia.org/wiki/Constructionism_(learning_theo...](https://en.wikipedia.org/wiki/Constructionism_\(learning_theory\)) .

Originally known for work on _Perceptrons_ with Marvin Minsky, AI researcher Papert later adapted theories from education towards the vision of "learning- by-making" and the (young) bricoleur [4]. This can approach can be seen in the evolution from 1960s graphical [turtle] [https://en.wikipedia.org/wiki/Logo_%28programming_language%2...](https://en.wikipedia.org/wiki/Logo_%28programming_language%29) to Lego [https://en.wikipedia.org/wiki/Mindstorms_(book)](https://en.wikipedia.org/wiki/Mindstorms_\(book\)) to modern day efforts to encourage coding-for-kids [5,6,7].

One of Papert's later collaborators, Sherry Turkel, discusses bricolage as it applies to programming -- [https://en.wikipedia.org/wiki/Bricolage#Internet](https://en.wikipedia.org/wiki/Bricolage#Internet) .

When it comes to early education, Turkel argues for epistemological pluralism [8] and cites anthropologist Levi-Strauss in comparing analytic science with a "science of the concrete".

We can appreciate both Seth Bling's concreteness [9] and Togelius's original papers for academics. Almost a decade ago, Togelius introduced Super Mario Brothers as a benchmark for reinforcement learning [10] and, with Karakovskiy, for AI more generally [11].

deepnet what's your interest in neural nets?

[1] [http://togelius.blogspot.com/2016/04/the-differences- between...](http://togelius.blogspot.com/2016/04/the-differences-between- tinkering-and.html)

[2] [https://news.ycombinator.com/item?id=16744694](https://news.ycombinator.com/item?id=16744694)

[3] For example, [https://scholar.google.com/scholar?&q=maker+movement](https://scholar.google.com/scholar?&q=maker+movement)

[4] [http://www.papert.org/articles/SituatingConstructionism.html](http://www.papert.org/articles/SituatingConstructionism.html)

[5] [https://scratch.mit.edu/](https://scratch.mit.edu/)

[6] [https://www.apple.com/swift/playgrounds/](https://www.apple.com/swift/playgrounds/)

[7] [https://github.com/google/blockly- games/wiki](https://github.com/google/blockly-games/wiki)

[8] [http://www.papert.org/articles/EpistemologicalPluralism.html](http://www.papert.org/articles/EpistemologicalPluralism.html)

[9] [https://www.youtube.com/watch?v=qv6UVOQ0F44](https://www.youtube.com/watch?v=qv6UVOQ0F44)

[10] [http://julian.togelius.com/Togelius2009Super.pdf](http://julian.togelius.com/Togelius2009Super.pdf)

[11] [http://julian.togelius.com/mariocompetition2009/](http://julian.togelius.com/mariocompetition2009/)</comment></comment>


Sophia v2.2 is out - <AUTHOR> https://groups.google.com/forum/#!topic/sophia-database/asQxturuGDw <comment><USER2> This is a bit stupid, but developers, each time you make an announcement like that, even on your own forum, blog, ..., take the time to write 1 sentence telling people what the software does. I clicked on the link, have no idea what it is and nothing made me want to click on the website to find out.

Even if it is the 5000th release, never miss that opportunity to acquire more users.

Same for web apps. One line somewhere at the bottom/top of the page to explain where people are and what the hell it is about. It used to be the norm 10/15 years ago, I don't understand why people miss one precious opportunity like that. Visitors aren't lazy, it's just that there is so many content to browse today. If you don't get their attention in the 30 seconds they initially spend on your page, that's too late.

It's the first time I hear about Sofia like many here, 95% of the audience that clicked on the link won't bother if they don't immediately know what it is for.

</comment><comment><USER3> If any of the sophia developers are here, please take note: your site is absolutely unreadable on mobile.

</comment><comment><USER4> How is this different from leveldb?

<comment><USER5> See the second paragraph of [http://sophia.systems](http://sophia.systems)

<comment><USER4> Leveldb does the same thing: [http://stackoverflow.com/questions/18447380/is- there-somethi...](http://stackoverflow.com/questions/18447380/is-there- something-like-redis-db-but-not-limited-with-ram-size)</comment></comment></comment>


Amon - web application debugging and monitoring toolkit - <AUTHOR> https://github.com/martinrusev/amon

<comment><USER2> This is the second or third time I've seen a service like this created (getexceptional.com and airbrake.io are two that come to mind quickly).

I can only hope this trend continues and in an open-architecture style. The ability to basically develop a skeleton monitoring UI that I can plug my own software into is fantastic.

My only thought is: why is this local to the machine I put it on? I'd rather have a central repository that I send me data to, like getexceptional. I don't mind having to set-up the process for each server I spin up, but I would prefer it if the information all aggregated to one source that I could browse through.

What I sort of imagine is registering for an Amon account and getting a unique key and identifier. Whenever I install Amon on a machine or want to call it from my code, I use that unique key and give the hardware a unique name and register my software service (pulling the hardware name from some config file automatically). Then, when I log into my central Amon account, I can browse through my exceptions and hardware details by machine name or software service.

Give me the ability to have multiple logging levels and I am in heaven.

Take it one step further: let fall-back be to either a local db or file-system and have a cron-job go through to try to fulfill the upload tasks.

That is definitely something I would pay $25-$50 a month for.

<comment><USER3> First off, some people have NDAs where they must not store any kind of file in the cloud, nor disclose anything at all (like the load of a server, graphs, the specs of the hardware, the used software, anything related to the company, its clients, its visitors or its services) to any third party.

So, as you may see, there's a point for this.

Also, don't forget that all you want for the replication already exists. You can easily replicate the mongodb data to another machine or back it up and save it somewhere else periodically.

As for giving each machine a unique id, this is a cool idea, but this implies that you have enough resources to set up your own monitoring system to monitor multiple hosts. That implies that you can implement something redundant.

</comment></comment><comment><USER4> Why do people keep linking to GitHub repos instead of official websites?

<http://amon.cx/>

<comment><USER5> Its nice to look at the source before you look at the actual application.

</comment></comment><comment><USER6> The installation and uninstallation scripts are idiotic, dangerous and clearly untested.

<comment><USER7> Would you care to elaborate?

<comment><USER6> Oh christ, where to even start..

First, it downloads and installs a non-checksummed i686 binary version of mongodb. This is outrageously anti-social and one should be embarrassed for releasing that. Ditto ez_setup and all that crap. This is 2011.

I'm only looking at the Debian script now, but it's littered with other bugs such as not calling update-rc.d for amon (it is called for amond). The suggestion to run "curl uninstall.amon.cx | sh" is particularly unsafe as the script has a Bash shebang and /bin/sh could be anything.

A glance at the rpm script suggests bugs fixed in one have not been copied over - for example, you use pgrep correctly in the debian script but in the rpm script you essentially do [ "$(ps aux | grep -c mongo)" = 1 ] which gives you completely the opposite result to what you want.

The uninstallation scripts are perhaps worse, riddled with typos ("/etc.init.d/amond", "torando", etc.) which demonstrate the code has never once been tested, deleting /usr/bin/mongo* even though you install it to /usr/local/ (and screw anyone who had their own version running), deleting any systemwide versions of jinja, and. It won't even uninstall on Darwin due to "[ "$(uname -s)" != "Darwin" ];". And you call "set +e" (a no-op as it's the default) so that if the site_packages shell expansion fails you try and delete "/jinja2", etc.

I'm going home now. This is just what I see on my first glance but I think I've seen enough, frankly.

<comment><AUTHOR> Thanks a lot for that.

 _First, it downloads and installs a non-checksummed i686 binary version of mongodb. This is outrageously anti-social and one should be embarrassed for releasing that. Ditto ez_setup and all that crap. This is 2011._

Amon is language agnostic, so there is enourmous chance, that a person who has 0 python knowledge could use it. By default none of the Linux distributions out there have pip or easy_install, so I thats why I have to install it.

 _I'm only looking at the Debian script now, but it's littered with other bugs such as not calling update-rc.d for amon (it is called for amond)_

That is for reason. Some people, especially on small VPS's prefer to keep their memory usage low, so that's why the web part is running on demand only.

 _It won't even uninstall on Darwin due to "[ "$(uname -s)" != "Darwin" ];"_

It's not running on MacOS at all at the moment, that's still work in progress.

</comment></comment></comment></comment><comment><USER8> Looks awesome.

I will try to deploy this on Heroku and see how it compares to my django- sentry [<https://github.com/dcramer/django-sentry>] installation.

</comment><comment><USER9> Looks very promising - I can't help wonder how you can spend time building a quality product like this, then release it for free : is there a monetization strategy at some point (i.e : pro support) ?

<comment><AUTHOR> There is a Pro version in the works. The idea is to have a fully working open source variant with almost everything included. But if you want to monitor multiple servers, better support for multiple apps ( for example: color filter in the web interface ), email notifications , etc - then you can buy the Pro version. The Pro version will cost something that everyone can afford and it will be - pay once, use forever :)

<comment><USER10> _pay once, use forever_

Don't hesitate to charge a reasonable monthly fee. There's a lot of room to undercut the existing players (newrelic et al) and still make it a sustainable business.

<comment><AUTHOR> Maybe I will do that, because it will be more convenient for some people. But the Pro version will always be an option for people that don't want to pay monthly charges and are OK with managing their own instances :)

P.S And of course the Open Source version will do the job for most people

</comment></comment></comment></comment><comment><USER11> I especially like the logging feature. Haven't had the chance to look at the code yet but how tightly coupled to mongodb is this? Would it be relatively easy to replace the store with redis?

<comment><AUTHOR> It was Redis once :) But Redis is not a good database for storing Rails exceptions, that's why I replaced it with Mongo.

<comment><USER11> :) Thanks for sharing your work. If you have the time could you elaborate on why Redis is not good for Rails exceptions?

<comment><AUTHOR> Because at its best Redis is an in-memory database. That means saving log information to it ( the exceptions are log information after all ) will take RAM. And at some point you will have to migrate to bigger VPS just because of that. You can manage that with the expiring keys, but some people prefer to keep their logs for long time.

<comment><USER11> Got it! Thanks for the explanation. Makes sense.

</comment></comment></comment></comment></comment><comment><USER12> Looks great, very impressive.

Is user authentication in the works? Is there an easy workaround for securing the web app?

<comment><AUTHOR> Almost there with the auth module, probably 2-3 days :)

</comment></comment><comment><USER13> Is this tool can be used with centtralized mongodb and app server to monitor more than one server ? I like its simplicity.

</comment><comment><USER14> One of the cleanest monitoring tools I've encountered. Looks promising.

</comment><comment><USER15> This is awesome!</comment>