From 1.5 GB to 50 MB: The Story of my Redis Database - <AUTHOR> http://davidcel.is/blog/2013/03/20/the-story-of-my-redis-database/ <comment><USER2> One of the long term things we want to address in Redis is lack of tools to analyze the dataset directly available in the Redis distribution, as first class citizens.

There are a few like redis-cli --latency, or redis-key --bigkeys (I encourage you to try both if you never did it before), but more are needed in order to profile memory, big keys, pretty-print the Slow Log output, and so forth.

The availability of external tools is surely of a great help, but I hope that in the long run we'll have the most important things integrated into redis-cli directly.

<comment><USER3> antirez - Author of redis-rdb-tools here. Would you be interested in a patch that adds profiling support to redis-cli?

My approach is to parse the rdb file so that the running server isn't affected. It approximates memory usage, but is fairly representative.

<comment><USER2> Thanks ksri, that would be very welcomed.

I think a viable design would be to put into redis-cli only things that are network oriented, so we should actually merge the functionality of your current tool into redis-check-rdb, that should be renamed just redis-rdb, with the following features:

1) Everything redis-rdb-tools is currently able to do.

2) Sanity checks of RDB files, like redis-check-rdb is doing already.

Makes sense? Thanks again, this would be very very appreciated.

<comment><USER3> Definitely makes sense! I'll start looking into this.

</comment></comment></comment></comment><comment><USER4> Great solution, but...

"Maybe I should revert to storing ratings in PostgreSQL and accept what would certainly be a large performance hit during recommendation generation."

I wondered if you prematurely optimized here. Did you try Postgres in the first place? What was the performance like? I can't help but wonder if you dismissed Postgres simply because it wasn't as sexy as redis.

<comment><USER5> To be honest, when I first wrote my site (and, later, recommendable) it was for my senior project in school. I had only been programming in school so far (which was fairly different from programming in the real world) and this site was really my first real-world application. Back then I was using MySQL (!) as my main data store. I was storing Likes/Dislikes as polymorphic Rails models, but then I read about Redis and it sounded like the perfect fit for all the set math I would have to do. So when refreshing recommendations, I would pull the ratings out of MySQL and feed them into Redis to do some fast set math. Then I'd tear down the Redis sets. I only had friends on the site testing it out for me, so I didn't notice much of a problem.

Later, I switched my database to PostgreSQL but wasn't having Redis issues at that point, so I let it be. Then when goodbre.ws found itself with a lot of new faces, I found myself with the I/O bottleneck I mentioned in the article. I decided to try to solve that by moving the ratings to permanently reside in Redis. That way I'd only have to hit Redis a couple times to regenerate someone's recommendations. Then I started storing too much in my ZSETS, but here I am now.

Now that I've reduced that memory footprint again, I feel as though attempting to move the ratings back into Postgres would be a premature optimization in itself. I was, however, looking into some combination of Postgres' array types (to replace the sets) and hstore (to replace the ZSETS, but sort in the query). I think it's worth looking into, but I have some other things to focus on first!

<comment><USER6> One thing that's commonly done is to keep the data in a normalized format, but use a view or a function to return the data in whatever format you want (arrays, json, etc).

This lets the application request the data in the ideal format for consumption. The database then becomes a "black box" service that returns a nested json graph -- the application no longer needs to know how the data is stored.

See <https://gist.github.com/joevandyk/5215014> for a simple example.

</comment></comment></comment><comment><USER7> If the values you are storing are integers, you should also look at your zset- max-ziplist-entries config setting. I've been able to shrink a 30GB Redis memory footprint to just under 3GB. The caveat is that all search operations in ziplists are sequential (since it uses variable-length strings and integers), but oftentimes scanning a short array is as fast or faster than hashing or locating an entry in a skiplist. There is more details about this here <http://redis.io/topics/memory-optimization>, and I've submitted a patch to Redis that makes it even more efficient (<https://github.com/antirez/redis/issues/469>).

</comment><comment><USER8> For relational databases, we already have some pretty good rules to avoid problems like this - the normal forms (specifically BCNF, which is "the" normal form to reduce redundancy inside a single relation). I wonder what rules apply to non-relational models? Has anyone done research into this?

<comment><USER9> Much of the research on network and graph databases from the 70's and 80's apply to today's non-relational models, at least from a logical data integrity subject area.

The short answer is that the "relational" model is a generic logical theory for managing data under a closed world assumption. "Non relational" mostly is about saying "I don't need a logical model, I need a persistence framework", wherein you are just looking to implement durable storage, but aren't really managing data in a logical manner.

Some network/graph databases are arguably a special case of the relational model, but they all vary to the extent under which they enforce logical integrity vs. exposing the innards to muck with (same with all RDBMS' frankly).

William Kent's 1978 book _Data and Reality_ is a great read to think of data modeling and integrity issues generically, irrespective of whether you use an RDBMS or a non-relational database. He highlights certain logical concepts as universal regardless of whether you use tables, K/V pairs, or graphs of binary relationships.

</comment></comment><comment><USER10> >What could possibly be causing it to use that much memory? I considered the length of my keys...Maybe Redis was eating memory by storing tens of thousands of really long key names in RAM? I decided to try shortening them to a more reasonable format: u:1234:lb for example.

Wow ... why would you reengineer your naming conventions without at least doing some back-of-a-napkin arithmetic beforehand and compare the potential savings? That strikes me as ... insane.

<comment><USER5> > Optimization is a rabbithole I've not had to go down very often. I am hardly > an expert.

But really, the answer to this is because it's easy to copy a database in Redis and then just do a mass RENAME. It took all of two minutes with a Ruby script and Redis' RENAME command. I could have done a theoretical back-of-a- napkin attempt, but I wanted some real numbers. And if it's easy to get them, why not?

<comment><USER10> Nevertheless, it was effort for dubious statistics (is knowing that the memory savings are exactly 10MB any better than estimating that the savings would be around 8MB to15MB, working with a data set that's 1.5GB big?). Anyway, it clearly doesn't really matter in this case. You're playing around with a side- project that you can just blow away anytime you feel like. However, when the calculations are this trivial, it's probably good to get into the habit.

<comment><USER11> Fermi estimates are a terrific technique, and indeed in this case we can see that "tens of thousands" x "five for each user" x 20B saved = 10MB, so that worked out nicely. So yeah, that would have been smart. BUT...

When Fermi estimates fail, they fail in one of two ways. Sometimes they fail by a constant factor, as expected, and that's fine. But sometimes they fail because you forgot one or more important factors entirely, and in that case, they can be arbitrarily wrong. (In fact, it seems pretty likely that the so- called Fermi paradox is an example of this). It's not just 8MB vs 15MB vs 10MB.... it's 10MB vs some chance of totally utterly wrong, like maybe 1GB or something.

So when the test for real data is cheap (as OP says it is in this case, not that I know squit about Redis), it's "probably good to get into the habit" of doing it right, instead of getting a perfectly good napkin dirty.

<comment><USER10> I know what you're saying but even if you're off by an order of a magnitude, at least it's some sort of starting point. Estimating that your key-space in Redis is between 5MB and 500MB isn't particularly useful but it is a big improvement over having no idea. I don't disagree with you that in most cases, there is no substitute for real world data, especially if you can come across it without much pain, but I maintain you should _always_ start with some sort of estimate, though wrong it may be. It's good to take the time to try to reason out a system or problem, and then seeing if your mental model matches the real world reality.

In this particular example the napkin analysis is so trivial and so accurate and the 'real world' data is so expensive ( linear look up of the entire key- space, followed by data destruction ), that it immediately jumps out at me.

<comment><USER11> When you forget even one factor, you can be off by many orders of magnitude.

And you're measuring the cost of the real data in totally useless terms. Who gives a shit about linear scans and "data destruction", lolz. What matters is how long it took the OP to figure out what command to write, and write it, and write the cleanup commands. If he already knew how to do it, then it was basically free. If he didn't already know how to do it, then he learned something, which pays for most of the cost of doing it.

And the napkin analysis is so trivial, but you don't EVER know that it's accurate, because there's no error bound on "oh I forgot the important part".

<comment><USER10> I don't think you know what you are actually arguing. I'm pretty sure I'm not arguing against performing real world tests to diagnose a problem. I'm pretty sure I'm not doing that because I'm not insane. I'm also fairly sure I didn't argue that you have to do one or other but not both. Though I didn't say it outright, I'm pretty sure I'm implied, and I stand by it, that it is good practice to at least do a fast mental (or napkin) estimate when it is warranted. Debugging is a hard process, but it shouldn't be a random one. You should have a mental model of your system, and it is good practice to go through the exercise of defining said mental model and yes, checking your assumptions, when it is warranted.

I'm clearly not a fan of the direction the OP took to verify his assumption that his key sizes have an impact on his Redis DB size. I think there's a better, more accurate way of checking his hypothesis (napkin math). His approach wasn't great and doesn't work for anything other than toy or test deployments, and the results aren't as clear as you may think (see below). I stand by that.

>And the napkin analysis is so trivial, but you don't EVER know that it's accurate, because there's no error bound on "oh I forgot the important part".

We're still talking about the same situation, correct? I suppose if it's pedantry you want, pedantry I can give. Tell me, how does the OP know that his "real world analysis" is correct? Because at some point in time, for some sort of input the system gave him one kind of result? Apparently a simple multiplication ( #of keys x avg. key size) is fraught with errors, but issuing a command (RENAME) against a black-box datastore the OP probably doesn't fully understand provides a clear, unambiguous result? What if Redis caches (in memory or to disk) all original values before issuing a key rename and then clears them out over a period of time, or better yet, doesn't clear them out until it needs the space? So you run your script, check memory usage, and see no difference.... so of course, because we're you, we trust "real-world result" and we live happily ever after ... yes?

Obviously this is a contrived example and OP most likely got the right result but I think I made my point. "Real world results" are full of gotchas and ambiguities (and sometimes require great deal of background knowledge to properly interpret), and in such a case it would be nice to have a mental model of what the expected result is so that it can be either verified or proven wrong (and thereby provide a direction for further investigation).

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> Always good to have a reminder to measure and validate your ideas of what is causing a problem before jumping to the solution.

</comment><comment><USER13> tl;dr version: his recommendation algorithm stored an amount of data that was Θ(n^2) + Θ(n*m) for # of users, beers. He optimized that to put a constant limit on the storage space used.

The clearest takeaway is: if you want to reduce the disk or memory footprint of your DB, you need to figure out what tables/rows/columns are consuming the lion's share of the space (there's almost always one column that's way worse than the rest) and figure out how to change your app logic to store much less data in that column.

The OP's lessons learned apply no matter which DB you're using!

</comment><comment><USER14> This reminds me of a situation I've seen and found myself in far too many times. When I encounter a performance problem I immediately think about areas where, during initial development, I stopped and said "this might cause a problem with a large amount of data" versus doing the proper thing of actually hooking up a profiler or doing proper testing to determine what the actual problem is; Rarely is it in an area I previously identified as potentially causing future problems.

Ego (I know what the problem is!) and the drive to fix problems quickly always get in the of finding the solution. Tales like this are a great reminder to not assume you know what the problem is.

</comment><comment><USER15> I was thinking you had switched from string keys to hashes, since I had read a post from antirez about hashes being more efficient for this type of thing. But as they say, measure twice, cut once. After all, profiling is the first step of optimisation.

<comment><USER16> Hashes can be efficient because redis compresses hashes that match certain restrictions (eg: under 1000 items, each item being under 1000 characters long) and the same can apply to lists, sets, and zsets.

This compression may have inadvertently been activated by the author when he pruned his zsets, which is why the savings are so apparent.

<comment><USER15> The primary reason for the reduction was going from ~7k items in the ZSET to 100 for several thousand ZSETS. That's not to say the built-in compression didn't affect anything, but obviously less data == less memory.

</comment></comment></comment><comment><USER17> Great read. Stuff like this is why HN is awesome. If I ever get into Redis memory/performance issues, will remember this.

</comment><comment><USER18> Thanks for posting this. The real-world optimization posts tend to be the most useful for me.

</comment><comment><USER19> I see a few good lessons here:

1\. You can learn a lot about your own code and your tools by digging in and debugging and experimenting.

2\. As macspoofing argues, having a correct mental model of your application, and your tools, is what separates professional programmers from amateurs and beginners. You don't need to do much math or understand Fermi estimates to figure out that removing 30 bytes of excess key name from, say, 100,000 rows is only going to get back a few megs. However doing the experiment validated your hypothesis and taught you something about how Redis (probably) stores keys internally. For many real problems with real data, estimating is orders of magnitude faster and simpler (and probably safer) than experimenting, so it's a skill to cultivate.

3\. This is exactly the kind of data management problem that relational databases are _really good_ at. Relational theory is based on sets and RDBMSs are good at working with sets. But you would need to normalize your schema and know how to write efficient SQL -- perhaps an exercise for another day. RDBMSs and SQL are not easy to learn and take real study and practice, but the rewards are significant. I agree with AznHisoka that dropping PostgreSQL (or MySQL) in favor of Redis was a premature optimization, but you would need to spend a lot of time mastering relational database design and SQL to get the benefits of an RDBMS, whereas Redis doesn't make the same demand. If you posted more details of your data and how you need to query it on stackoverflow you'd get some free lessons in schema design and normalization.

4\. A database of a few tens or even hundreds of thousands of rows and taking up only 50 megs of RAM is trivial for any RDBMS. Not trivial as in not important, but in the same sense that delivering a carton of milk with a semi truck is a trivial use of the truck. Your data set would not begin to run into any performance limits of a stock RDBMS. I'm not criticizing or making fun, just stating a fact.

5\. Don't assume you know what the problem is when debugging or optimizing -- it's too easy to go into the weeds and get frustrated. Over time you'll probably get better at narrowing the range of possibilities but even the most experienced programmers are frequently wrong about the source of a bug or performance problem. Your methodology is correct, do the same thing next time.

You say that you are relatively new to programming and databases, and there's nothing wrong with that. We've all made the same kinds of decisions you have, and some of us are still making them without the kind of introspection you've documented.

If your code seems too complex (something you develop a feel for over time) you need to choose better data structures. With the right data structures the code writes itself. When you speculate about using an array column type in Postgres I know with 99.99% certainty that you are not normalizing -- nothing you've described can't be accomplished with normalized relations. And you'd be able to do all of your queries with simple SQL statements.

I recommend Chris Date's book "Database in Depth: Relational Theory for Practitioners," which is aimed at programmers rather than DBAs.

I've written more on this (and linked to some more good books) on my site at <http://typicalprogrammer.com/?p=14>

</comment><comment><USER20> Hurry for real-world optimization!</comment>


Django Vanilla Views 1.0 released. - <AUTHOR> http://django-vanilla-views.org/ <comment><USER2> Contrary to somewhat popular belief, Django's function-based views are not deprecated and as far as I know, there are no plans to remove them. Django's class-based views were introduced and the generic views ported over to the CBV style (the older FBV versions _were_ deprecated), however after giving it a go in a small part of our app, I found them to be overly-complex and hard to extend and went back to FBV.

In my opinion, Django's CBV's are polarizing. Some people really love them as a way to reuse code (e.g. they are recommended glowingly in the popular Two Scoops of Django book) and people like me don't. I feel like inheritance is a poor way of modeling views and reusing code.

Whenever anyone voiced complaint about the CBV system, it usually was mixed in with complaints about the implementation of the generics. The response was always, "you don't have to use the generics provided, write your own." I'm really glad to see that Tom went exactly that route, and I'm happy that Django as a system is flexible enough to allow it. I may not be 100% convinced just yet but I'm willing to try this out and it seems like a definite step in the right direction to me. Nice work.

<comment><USER3> I have been looking at them (but not using them) for a while because I cant see why I would want to re-use a view + it seem s like they require more work to implement then

if request.method == "POST":

       return render_request('template.html", params)

<comment><USER4> The basic concept doesn't make a big difference. The real meat is the generic model views, not the base CBV, imo.

It may not be revolutionary, but extending an UpdateView to block anonymous users and throwing a mixin that limit access to the objects the current user owns, I can have my "edit a blog post" view looking like:

      class EditBlogPost(OwnedObjectsMixin, AuthenticatedUpdateView):
        model = Blog
        form = BlogForm

and all I have left to do is put my template, which is quite nice (especially when the List, Create, Delete views are about the same length in code).

It gets less useful (and more convoluted) as your views get further away from basic CRUD stuff, but I still feel it's quite useful.

(Edited: the difference is more clear on the "Update" view)

<comment><USER5> As opposed to two decorators on the blog module's edit method? I'm not sure a ClassView grants us any advantages for this use.

      @is_authenticated
      @is_owner
      def edit(self):
        ...

<comment><USER6> The decorator can only add behavior prior to the original function's execution, and many decorators may not necessarily be applied together. CBVs allow a composition of mixins that can include behavior that's substantially more complicated and can deal with any of the methods and members defined for the view.

A well-defined base CBV allows you to limit the use of RequestContexts, checks for method types, and special validations. For small projects it may not make a difference; for projects with at least couple dozen views and a bunch of common behavior, they increase readability tremendously.

Also, they offer a very good template for defining view behavior. Once you're familiarized with CBV method definitions it's much easier to grok code, as the view flow is much more standardized than for regular function views.

</comment></comment></comment></comment></comment><comment><USER7> This should be the standard. Today, I'd argue that class based generic views are the most complex part of Django to learn. The concern is usually met with rather blunt responses like 'read the source code' or 'try harder'.

The goals of having a class based generic view was quite appealing with the possibility of leveraging inheritance and properties. But Mixins were probably the wrong way to achieve it. The Method Resolution Order can get very confusing leading to hard to resolve bugs. It seemed to be a very un-Pythonic ("If the implementation is hard to explain, it's a bad idea") implementation to me.

<comment><USER8> > I'd argue that class based generic views are the most complex part of Django > to learn.

Also, after using them in a few projects, they reduce grokability and maintainability. Django's ORM is much more complex, but it operates at the right level of abstraction. CBVs always felt a bit too high level to work well. Going back and reading CBVs I wrote months ago always took more time than it feels like it should have.

That said, it took the ORM a while to get right, too, and the Django community is thoughtful when it comes to abstraction, and I'm sure views will land in a good spot.

</comment></comment><comment><USER9> I completely disagree with pundits that argue CBVs are way more complicated than function-based views.

Three key differentiators for me are the reduction in redundancy with CBVs, consist organization of code and better code reuse through inheritance. This comes out best with generic model views (as one poster has said).

The real problem with CBV generic view usage is that it's not very well- documented, specifically the order of method calls. I often have to refer to the source to understand the flow (which is fine, but a pain, and I imagine extremely intimidating for new users).

Were this well documented, I think it'd actually be _easier_ for people to get into Django - because CBVs outline a lot of the key features that people would use in Django anyways, except now in a much more formal, easy-to-use way (i.e., get_context_data, Form submission and so forth).

In contrast, function views seem to be more for experts who need to do some kind of complicated view processing that goes beyond the bounds of a typical CRUD app.

</comment><comment><USER10> I'll paste my thoughts from a conversations I had with someone about them:

"There are places where the generic views could/should be simplified, get_template_names and get_form_class are horrendously complicated. But his big example is removing get_form_kwargs, which is probably my single most heavily-used function when using them (Django's generic class-based views). Overriding form_valid is always a little gnarly, because you need to know the implementation details to know it's not worth calling super."

In general, I'm not sold on Tom's specific implementation, primarily because it throws away parts of the API that I personally use quite heavily. Of course, the counter-argument is that he's kept the parts that he uses heavily. I suspect the ideal solution is somewhere between Tom's implementation and the existing one.

<comment><AUTHOR> Hi Andrew, the migration notes cover the removal of `get_form_kwargs()`.

The simple answer here is that you're actually much better off just overriding `get_form()`. It's no more complicated, and it's much more direct and obvious. We can do away with `get_form_kwargs()`, `get_initial()` etc - they're unnecessarily granular.

> you need to know the implementation details to know it's not worth calling > super

The simplicity is self re-inforcing here - the implementation is trivial, so it's easy to figure out how to totally override a method if you want too. Of course this doesn't prevent you from using a `super` style if you want to.

<comment><USER10> There's something about your example that wasn't convincing to me, but I've tried it out with some of my use cases. Whilst your approach is less DRY, it does seem to lead to more readable code, so I'm going to look into it further.

We should fork django-extra-views and see what we can come up with.

<comment><AUTHOR> > We should fork django-extra-views and see what we can come up with.

Yup, that'd be interesting. Hopefully you've seen my comment to that effect here: [http://django-vanilla-views.org/topics/django-extra-views- co...](http://django-vanilla-views.org/topics/django-extra-views- compatibility)

<comment><USER10> A little amusing that the only bits that work are the bits I've been regretting merging in :)

</comment></comment></comment></comment></comment><comment><USER11> I started working with Django in 1.3.1 as my first REST framework. When I got to the views, it was easy to understand how the function responded to an HTTP request:

if request.POST:

validate

save form

redirect to success page

else if request.GET:

show the form

Learning the views this way was very helpful when I barely understood how the internet worked. So naturally, when I took on an intern this year, I got her started with the Django tutorial, then gave her the above pseudocode as an example of how to write a simple form. After she worked on it for a while and got very confused, I looked over the Django 1.5 tutorial with her and saw those awful, overmagicked generic views they use now. I spent an hour trying to implement what would have taken me two minutes if the views were written manually, until I gave up and rewrote all of her code while standing on my soapbox.

Needless to say, I've been hoping for something like this project ever since.

</comment><comment><USER12> Django class based views always seemed overly complicated to me. I cope with it thanks to the useful [http://ccbv.co.uk/](http://ccbv.co.uk/) and frequently had to hand draw the inheritance tree to understand what the hell was going on with my method calls.

I'll be curious to know what was the actual reasoning behind this complicated design.

<comment><AUTHOR> I thought the design seemed reasonable at the time, the strict separation of concerns that the mixin classes provide is valid enough.

It's only now that we've been using CBVs for a while that it's become more apparent that they're somewhat over-designed, and can be awkward to get to grips with.

</comment></comment><comment><USER13> Note to non-regular django devs: a django view is equivalent of the controller in other frameworks.

<comment> X-Istence Not the case for Pyramid for example...

</comment></comment><comment><USER14> Interesting

\- Django regular views are ok (though not class based)

\- Django "class views" are a product of brain damage, really (see the example to see what I mean)

This looks like sane class based views, but I was expecting something more like the way Pyramid does it (which are not generic, but class based)

<comment><USER15> > Django "class views" are a product of brain damage, really

"Brain damage" is rather harsh: they're following a valid design principle but (IMO) carried a little further than the scope of the problem might require, running into the classic tradeoff versus ease of acquisition.

</comment></comment><comment><USER16> My experience has been that plain old function based views work out to be simpler and more flexible, at the cost of a little more duplication. Don't forget they are an alternative too!

<comment><USER17> Plus you can mix and match different view styles. If you have a lot of very similar views, elevate them to class-based views, or a smarter function-base view. If you have a lot of dissimilar views, you can keep them in function- based views while other parts of your app are using Vanilla Views or class- based views. Mix and match as appropriate (right tool, etc).

</comment></comment><comment><USER18> I'm working on my first large scale programming project, learning Python and Django right now. I am in the midst of trying to figure out CBV's, and I am starting to understand and agree with the criticism of the generics that commenters have been making. If I want to make any modifications, I am finding myself just writing my view from scratch. I'm going to give this project a try today and see if it makes my life easier!

</comment><comment><USER19> I personally don't find the regular CBV to be difficult to use and I prefer CBV to FBV in 99% of cases. I actually find it _easy_ that the flow is separated into different methods, so that when I want to change a specific behavior, I don't have to read the whole view. That being said, I do find that I have to look at the source code for CBV from time to time, even though the documentation is quite good - but maybe "looking through source code" isn't quite as intimidating to me as to more beginner programmers.

</comment><comment><USER20> I guess I'm in the minority - I really like the class based views for simple crud work. Obviously function based views are a better fit for things that are outside of the simple. but I've had a lot of luck using CBV to simplify crud work.

Interested to try out django vanilla views

</comment><comment><USER21> Side comment: that's a nice website; how did you make it? I see it's hosted on GitHub pages and that you use Bootstrap. Did you write all the HTML/CSS yourself, or use Markdown, etc?

<comment><AUTHOR> Thanks!

The documentation source is markdown.

The styling and documentation building script is custom, and taken from the documentation for Django REST framework.

I'd like to build it into a proper markdown docs tool someday if I can find the time.

<comment><USER22> Dear God, this. I started looking at your makedocs.py over the weekend myself. I've never liked rst.

</comment></comment></comment><comment><USER23> In the example where does the account variable come from in the form_valid method?

def form_valid(self, form):

            send_activation_email(self.request.user)

            account = form.save()

<comment><AUTHOR> Hiya, I think you've misread the example. The `account` variable doesn't come from anywhere outside the method. It's being instantiated right there.

</comment></comment><comment><USER24> This brings Django one step closer to including scaffolding (like Rails).

<comment> X-Istence scaffolding doesn't require class based views ...

</comment></comment><comment><USER25> License isn't open. Pass.

<comment><AUTHOR> Uh, whatcha talking about? It's a standard BSD 2-clause license.

[http://opensource.org/licenses/BSD-2-Clause](http://opensource.org/licenses/BSD-2-Clause)

<comment><USER25> Doh. My bad.

<comment><AUTHOR> No problem :)</comment></comment></comment></comment>


Manufacturing and the Great China Firewall - <AUTHOR> Hi All, ShinePay factories are running proprietary software to enable manufacturing operations in China. We're having a difficult time maneuvering around the great China firewall. If you have any experience with this I would love to bend your ear. Please & Thanks in advance.<p>VPN is working via the browser but not via python with requests library. ping in terminal doesn't work too. But they can access my AWS hosted server via the browser <comment><USER2> If the exact same request is working in the browser but failing for the Python program, they're probably not tunneling all traffic on their computer through a VPN, but have only configured the browser to use a proxy. You should be able to make the requests library use the same proxy. IIRC, there's an environment variable you could set to achieve this without code changes.</comment>


Writing programs using ordinary language – MIT News - <AUTHOR> http://news.mit.edu/2013/writing-programs-using-ordinary-language-0711 <comment><USER2> Already, above the fold, you see the problem:

       Text Description:
       three-letter word starting with 'X'.

       Regular Expression:
       \bX[A-Za-z]}{2}\b

"été" is a word too. Without going out of English, "'em" is an English word, granted, not starting with a X, but demonstrating that letters are not [A-Za-z].

So once you've specified what you really meant:

       Sequences of three characters, the first being 'X',
       and the two others being any letter in the set
       "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
       (and therefore excluding accented letters, ideograms,
       greek, russian, etc).

you realise that you really want to be able to write \bX[A-Za-z]}{2}\b instead. DUH.</comment>


Ask HN: Typekit for website backgrounds? - <AUTHOR>

I'm looking for a service that offers curated backgrounds for websites. Typekit for backgrounds. Stock image galleries contain too much garbage.<p>Does this exist? <comment><USER2> <http://subtlepatterns.com/>

<comment><AUTHOR> This is a great resource. Do you know of any libraries with photo backgrounds?</comment></comment>


Ask HN: Good examples of software engineer portfolio websites - <AUTHOR> I am a software engineer, and I would like to make a personal portfolio site.  Any great examples? <comment><USER2> Resume or portfolio. Be creative and clever. Be unique.

These are some ones I like:

[http://themes.jsonresume.org/theme/elegant](http://themes.jsonresume.org/theme/elegant)

[http://kalynnakano.com/](http://kalynnakano.com/)

[http://michaelaldridge.com/portfolio/](http://michaelaldridge.com/portfolio/)

[http://www.garysheng.com/](http://www.garysheng.com/)

[https://rafaelderolez.be/](https://rafaelderolez.be/) ( I am definitely a fan of the no scroll page)</comment>


Linux Kernel Prior to 5.0.8 Vulnerable to Remote Code Execution - <AUTHOR> https://www.bleepingcomputer.com/news/security/linux-kernel-prior-to-508-vulnerable-to-remote-code-execution/ <comment><USER2> [https://www.reddit.com/r/sysadmin/comments/boiknf/linux_kern...](https://www.reddit.com/r/sysadmin/comments/boiknf/linux_kernel_prior_to_508_vulnerable_to_remote/)

[https://www.reddit.com/r/netsec/comments/bofpri/linux_kernel...](https://www.reddit.com/r/netsec/comments/bofpri/linux_kernel_prior_to_508_vulnerable_to_remote/)

[https://www.cert-bund.de/advisoryshort/CB-K19-0400](https://www.cert- bund.de/advisoryshort/CB-K19-0400)

</comment><comment><USER3> Can anyone comment on when this bug was first introduced?

Prior to 5.0.8 seems a little vague, or is my 2.4 box actually affected?

<comment><USER4> a more exhausting list of Kernels affected: [https://www.securityfocus.com/bid/108283](https://www.securityfocus.com/bid/108283) (the link is in the article)

<comment><USER3> Thanks for the link - though I really couldn't find it in article. Doing a quick Ctrl+F on the source code doesn't show securityfocus being linked to at all.

Edit: HolyCarp: it looks like my ancient 2.4 kernel actually is vulnerable!

<comment><USER5> This looks more like an automatically generated list of all releases "prior to 5.0.8". Since this vulnerability is apparently connected with net namespaces I doubt it goes all the way back to 2.0.

Edit: git blame shows this commit introducing the if statement that is touched by the patch linked by OP. It happened somewhere in the 4.2 branch. This doesn't mean that the problem was introduced exactly then, but it does show that it's relatively recent code.

[https://github.com/torvalds/linux/commit/467fa15356acfb7b2ef...](https://github.com/torvalds/linux/commit/467fa15356acfb7b2efa38839c3e76caa4e6e0ea#diff-3604e3582e525cf360833fba0165fa3eR357)</comment></comment></comment></comment>


Utah man who runs popular YouTube channel is charged with possessing explosives - <AUTHOR> https://www.sltrib.com/news/2018/01/18/utah-man-who-runs-popular-youtube-science-channel-is-charged-with-possessing-explosives-during-backyard-experiments/ <comment><USER2> I bet this shit wouldn't happen if the local government were filled with Libertarians...</comment>


Ask HN: Should we join accelerator run by a company in same market if invited? - <AUTHOR> <comment><USER2> Is the accelerator any good? What kind of track record do they have? What kinds of mentors are available to you? How much equity do you give up?

<comment><AUTHOR> They are very new and I do not see a lot of reviews. The equity ask is on a higher side (5% for 25k-50k EUR). They seem to have made many recent investments(claim is 100) but in Europe(We are in US).

<comment><USER2> Would you feel comfortable reaching out to some of their portfolio companies to learn more about their experiences? I would be just as (if not more) worried about the quality of the incubator than about the fact that it is owned by a competitor.

<comment><AUTHOR> thanks! I just did.</comment></comment></comment></comment>


Show HN: wa-automate v2.0.0 released now with CLI - <AUTHOR> https://github.com/open-wa/wa-automate-nodejs#cli <comment><AUTHOR> Ever wanted to turn your WhatsApp account into an API? Now you can with one command

> npx @open-wa/wa-automate --help</comment>


Java Hack: Double Brace Initialization - <AUTHOR> http://www.refactory.org/s/double_brace_initialisation/view/latest

<comment><USER2> Watch out - double-brace initialization is cool looking, but it's orders of magnitude slower than regular initialization as it must generate an anonymous class.

Also, the generated class is put into the permgen space, which is not garbage collected. The permgen is pretty small by default - and if you fill it up, your system is hosed.

I use them all the time in unit tests, but never in production code.

<comment><USER3> The compiler generates the anonymous class. To the runtime, isn't it just a class with a name given to it by the compiler, not a developer (question mark...my key broke today)

It's true that it uses up permgen though, but if it only adds an initializer we're not talking about much space.

Edit:

I did an experiment. This is the code:

         new ArrayList<Object>() {
           {}
         };
         new ArrayList<Object>();

Here is the bytecode:

         0  new Main$1 [16]
         3  invokespecial Main$1() [18]
         6  new java.util.ArrayList [19]
         9  invokespecial java.util.ArrayList() [21]

And, the class:

        class Main$1 extends java.util.ArrayList {

        // Method descriptor #6 ()V
        // Stack: 1, Locals: 1
        Main$1();
          0  aload_0 [this]
          1  invokespecial java.util.ArrayList() [8]
          4  return
        Line numbers:
          [pc: 0, line: 10]
          [pc: 4, line: 1]
        Local variable table:
          [pc: 0, pc: 5] local: this index: 0 type: new Main(){}

Just a regular old subclass.

<comment><USER4> You can always copy and paste a question mark (?). I don't recommend coding that way, I once tried coding with the left quarter of the keyboard broken, not fun.

<comment><USER5> alt + keypad 63 works too (at least on Windows)

</comment></comment></comment></comment><comment><USER6> For initializing lists there is a less verbose expression in Java:

      List<Integer> list = Arrays.asList(1,2,3,4,5);

<comment><USER7> . . . except that doesn't get you a modifiable list, it just wraps a List interface around the array that's implicitly created by the var-arg, so it's a trick you have to be careful with.

Personally I often end up writing little helpers like: List<T> list(T... args) { return new ArrayList<T>(Arrays.asList(args)) } usually in test code. A built-in List and Map-initialization syntax is really something every decent programming language ought to have.

<comment><USER8> (Consider using the Google collections library if possible. It already has that in the form of classes like Lists, Maps, etc that provide builders and factory methods.)

</comment></comment></comment><comment><USER9> Nice. Could have used that thought about 9 years ago :)

</comment><comment><USER10> This is where Google Collections comes in handy:

        Lists.newArrayList(1, 2, 3)

Or, if you don't need to modify it:

        ImmutableList.of(1, 2, 3)

Such an awesome library.

</comment><comment><USER11> I've been using this for years. It is a fantastic way to build up hierarchies. XML, maps, JSON, UI code etc. It is very concise, fast and produces clean code which also just happens to correctly indent when you apply code formatting.

      School s = new School() {{
        add(new Student() {{
          name = "Bobby";
          age = 15;
          add(new Pet() {{
            name = "Fluffy";
          }});
        }});
      }};

      vs.
      School sc = new School();
      Student st = new Student();
      st.name = "Bobby";
      st.age = 15;
      Pet p = new Pet();
      p.name = "Fluffy";
      st.add(p);
      sc.add(st);

</comment><comment><USER11> One thing though. I love this technique but the home built Eclipse compiler is not fully Java language compliant and it will reject certain forms of this as invalid. Specifically, the sun and eclipse compiler treat "this" differently in these code blocks. My project will not compile under eclipse because of certain usages of this pattern when combined with inner classes. The bug crops up when you need to pass the parent to the child object being initialized in the static block.</comment>


Two big sunspots are staring directly at Earth - <AUTHOR> http://spaceweathergallery.com/indiv_upload.php?upload_id=127438 <comment><USER2> Whenever I see close up pictures of the Sun, I'm always blown away by the scale of them. Each one of those granules is ~1000 miles across, and the Sunspot itself is around Earth size in scale. We think of our mountains as huge, and they take millions of years to grow and die. Features on the sun are truly massive, and they are constantly changing, lasting hours to days.

</comment><comment><USER3> what does this mean? is there any context? Thanks!

<comment><USER4> [https://en.wikipedia.org/wiki/Solar_flare](https://en.wikipedia.org/wiki/Solar_flare)

[https://en.wikipedia.org/wiki/Coronal_mass_ejections](https://en.wikipedia.org/wiki/Coronal_mass_ejections)</comment></comment>


Ask HN: What would make a great customer support tool? - <AUTHOR>

Hi,
I keep seeing posts on HN about customer support (email support) tools. We have been using Fogbugz for a couple of years now and I know that there can be a better solution. I have looked at a few other solutions (for example zendesk and RT) but I am wondering what a great customer support tool for a small team should be like. I have a few ideas
* Ability to reply/close/assign and do other stuff through email (RT does some of this but its extremely hard to setup)
* A very simple and usable web and mobile interface with a clean API
* Pricing based on number of tickets and not number of users (not all developers are support agents but you still want a login for them to see/respond to some tickets)
* Hooks to provide more information about the user, making it easier to provide a more personalized response. Basically making it easier to wow people who write to you.
I feel a lot of solutions out there are pretty complex (in terms of features, integrations etc) and quite costly for small teams. Do you think there is scope for a new product here? What would you want to see in it? I am going to start working on something very soon and would appreciate any feedback or ideas. Also, would love to find some beta testers :)
Thanks! <comment><USER3> There's definitely room for such a product... which is why I formed a company with a friend and wrote one. :)

Our product is currently in closed beta as we squash bugs, but we hit every bullet point on your feature list, plus a few more that we've uncovered as our customers use the system.

Of course, we use this for all of our bug-tracking and customer support as well, and it's amazing what having a _good_ support system does for your workflow.

So, we're competitors, but don't let that stop you. After all, competition is the crucible in which truly amazing products are formed. :)

<comment><AUTHOR> Thanks for the encouragement. Yep .. healthy competition is always great and exciting to have :)

</comment></comment><comment><USER4> I've thought a lot about this and sketched out some ideas myself. I work at apartmenttherapy.com -- which is primarily a content/advertising oriented business, though we are building toward some more app & community-oriented features in the near future. Most of the systems I've evaluated tend to fall into either a "bug tracker," "help desk" and/or "CRM" bucket, none of which is a good fit for what we need. What we need is more of a "reader communication" platform. We get a LOT of incoming email but we don't have very good tools for really tracking it. Options I've evaluated are generally too rigid and expensive, or flexible but a lot of work to adapt & maintain , and ugly (e.g., RT).

Being able to manage the system solely over email, at least for basic interactions, I think is really important for my team. I really liked some of the features of Email Center Pro (<http://www.emailcenterpro.com/features/>) but found their UI a bit too clunky and the inability to do basic management directly in the email client was a killer. A streamlined/updated version of that plus email integration similar to RT would work really well for us, I think. If you want to chat more, please email me scott AT apartmenttherapy.com.

<comment><USER5> One of the ideas was to create something like a local business social network and use it for connecting employees and customers. You would ask your users to login with their Facebook account to your system (benefit - a lot of info about the customer) and then hook them up to the product they have and deal value they generated. This would give you good info about importance of the call.

Idea could be summed up as "Quora for support with CRM integration".

<comment><USER6> I think the problem with Facebook, is in a straight B2b job, quite apart from the fact i think i could never legally clear working with FB apis, is the info people put on their FB is not relevant to their work. The info i want to plug in to is such as reporting structure, tenure in a position, line responsibility, projects, published papers, conferences attended or talks given. That's very specific stuff which sometimes is publicly available, but usually not. THis intel comes off the direct sales floor. But it can be super useful when you need to work with a customer to understand the scope of their needs, because you have some kind of map based on which you can (if you are good at this kind of interpersonal) start asking how your bit of code affects group a in a related department, or if a change to say a account presentation will go down well with CFO. There are commercial databases which try to keep up with personnel changes in bigger companies. Thomson does one. "Amadeus" a Belgian data source does well for company reporting structure. I'm a firm believer in selling features up the chain in a customer site, not "upselling" them new products. Dones well, this has sometimes had for me the side - benefit of locking consultants out of the loop. Nothing worse than aiming a project or sub - project at needs you've discovered at a custmer, to have a different management branch bring in a consultant to "evaluate" your work's worth. So, i try to insist that the primary role of sales is not necessarily to be closing deals, but to be closing the data we need to work quickly with a customer. I'm specific too, we never use the word "client" beause it sounds too open ended, too "maxing out billable hours". I try to teach sales that the efficacy of their sale is directly related to how easily we can start to work as closely as is practical or legally appropriate with a customer. Basically, play the consultants at their own game. You can bet they know who is who in a attractive potential customer site, because, generalizing,that sems to be the only leverage they ever have.

It's a nice idea you have there. But it's 20 years since i first experienced customer receptions declining to give any info as to employees, even for specific roles / names. I've no idea how legal in a big corp would think, about even using FB personal data to approach or manage contacts with employees. That said, the first thing i learned about sales, which is still an essential component of your job, i'd say particularly so if you're coding to fix a customer problem or desire, is to verbally map out common contacts with the first person you speak with. That's not far from what FB actually does, but you're doing it in person, albeit down a phone call often as not, and so concerns as to privacy can be finely guaged if you learn to listen.

The kind of data we pull in for any company we'd like to wok with is theses, lectures, of who we think pays attention to what we do, even if not "decision maker" then we delve deeper to try to understand the scope of a department influence - are they a skunkworks, with board support, are they mavericks, are they hard pressed ops guys? Sometimes just because who you communicate with might be classified as one of those types, you may be doing a lot of help by widening your contacts outside, because for big corps it's not a given teams talk to teams talk to the right management layers all the time.

What i'm on about i think works for any company you work with above 50 employees, and requires a lot of discretion. I don't have any "hard sell" function in where i work, but it's good to learn off experienced sales guys, and quite often they've worked "hard sell" before. My definition of "hard sell" is to just be absolutely direct about what you want to convey, waste the least time of any party, and haggle price especially if your bit of the project can give longer term or wider benefits. My own experience is you're lucky if you an pitch a technical project at a senior manager and get two way conversation, so the idea is you mix up who you speak with. Learn off a sales guy how to get past secretaries, also, it is handy in a pinch!

</comment></comment></comment><comment><USER5> I was discussing a new kind of support system with my friend the other day. We have been trying to understand current limitations of the support systems and if they can be fixed with technology (not always).

I'm looking forward to see some comments from people that are more into this topic!

<comment><USER7> I feel that a lot of the points made by the original poster are just bolt on features and not actually evolving support systems. Sure, they are nice improvements, but I think I've seen them all at one time or another. Like you said, not all problems can be fixed with technology. At least, not with the base email support systems we have now.

I'm more interested in seeing some out of the box thinking.

For example, build in some AI to preselect/recommend a base response from a pool of previous responses. That doesn't mean it'll be perfect (and still needs to be personalized), but it's better than starting from scratch or scraping through your wiki looking for a previous response.

Also, I want a view that watches an email support queue, public forums and possibly chats. That way my support reps can take the next available request no matter where it comes from.

<comment><USER5> Yesterday we had a discussion and following point we made: \- nothing that we can do (simpler UI, faster search for the knowledge base etc.) is good enough to take it to market. People simply will not convert to our app because it will not fundamentally change the way they work - and help them fix their problems. We will still do the same thing, only a little bit better then competitors.

Also we joked that if a company want to make a user happy we can setup an engine that will send emails randomly from around 100 different templates assuring user that people are working on his case ;)

<comment><USER6> You say no-one will convery because your app is not fundamentally changing how they work.

How about you get really intimate with what they use, how they use it, and work on simplifying workflow UI, readability, all the human factors stuff which seems to be overlooked once a project gets rolled out because the painful bit of codifying processes too all the budget?

I mean, work out what you could overlay as a benefit, in a drop - in app. Ignore bling, save where used to flag data that needs working on urgently, and think about that old hullabaloo of the 70's consultant with clipboard: time and motion. Think how many times a UI context switch is required. Take away frustrations.

Obviously i've no idea if what yu're working on is the slightest bit suitable fr this treatment. But if i had a nickel for every time i've seen a support rep slam down their phone (i was going to say smashed, but we're all enlightened PC people now!) because of something as dumb as having to hit the fourth button on their machine to transfer a call, i'd be doing pretty good.

It's not fair of me to keep using phone use analogies, but i don't se enough info as to what other people ar trying to solve here, hope you get my drift anyhow.

Can you write "helper app"? Remember multifinder or i forget the name but whatever it was on early macs which allowed you to copy and paste to a stack? Id there something functionally simple like that you can offer which doesn't mess with the underlying interfaces, but which still requires some pretty neat writing and likely a fairly deep understanding of the existing apps to make work consistently and without snarling anything?

Whenever i get private project time, increasingly rare sadly, i keep sketching out UI models for a desk phone which would be complimentary to the tracking / contact apps i use, not be a awful ugly alterntive. Things like flipping a contact window from the list on the handset to my main monitor. The hard bit isn't so much now i have to work out a way to communicate that from phone to PC, because one way would be to loop the phone data to the PC and use the new USB screen driver tech. The hard bit is working out what you want to really happen. Is that, you finger flip a contact towards your main screen, and you get a fully expanded list of the discussions you've had with them? Can drag and drop work with this in a kind of wacom tablet way, so then hw does movement on the obviously smaller touchscreen translate to your main monitor? If, say, i've a preference for greyscale display on anything which cannot be usefully colored to convey real information, does the "flipped" window stay greyscale? Do i want the flipped window to be large, float or be even a normal window, or do i want it "brought back" / closed by a gesture on the phone screen? How could i do that from a cell phone?

That's pretty pie in the sky for a solo project, but whilst there remains good utility from desk bound objects (my favourite gripe is i can dial important customer numbers from memory faster than i can bring their entry up on my very latest processor Nokia) and these objects, like everything else, wear out, and are now nicely converging to voip protocols which are mature enough there's no reason to go for a proprietary type such as cisco, theres a market for these things.

Making any of this elegant is no small ambition. But consider this thought, that i remember new employees getting "phone training" because they needed to know how to handle the butt ugly interfaces of pbx attached phones which to this day are not often much better than a bunch of soft keys and if you're lucky those keys are customisable but still very basic features. Why can't i drag and drop a call with my finger whilst i'm on another call? The other thing is with the iphone we really have a lot of people finding new ways to use a telephone, and hat idiom i think will spread to the desk phone. Suddenly it may not be necessary to train someone how to put a call on hold, transfer and pick up the next incoming without muffing it with fat fingers. If i remember, AT&T research got quite far as to graphical interfaces, i think the device was called an I-O or something. It ran on an ATM network for the prototypes.

I can;t quite be apologetic about my phone obsession. Companies still buy an awful lot of desk phones, and getting those to work with even simple contact apps is often a total PITA, and not much benefit is derived. Make the right connector, and you've a very interesting straddle market, selling forklift hardware upgrades and the back - end to make them play nice. Ouch, i just remembered that awful word "middleware"!

About the time they ran those Apple ads where some exec thought his staff were taking office computers home, but instead was told they were bringing their macs to work, i worked front line call center as a first job, where the productive people brought in their own phones and headsets, and went to extremes to patch them in to the corp pbx. That time, the phone of choice was a AT&T model with a 4 line VFD which we could read even in bright light and at an acute angle, and which you could "alpha tag" your incoming numbers, basic caller identity, but which the official office system supposedly didn't support, but did, and we hacked (nicely) our own caller groups so we could take decent length breaks and be covered by a buddy across the room. Make that stuf real simple, and - apart from the up tight "Office the movie" kind of echoes this gives off, yu have a lot of people in regular jobs wanting your kit badly.

Idle random thoughts, really. I think this pretty uncool bit of daily work is simply under cared for. Yet there are an awful lot of very big businesses for which the phone is their primary income point. Maybe if this ageing tech actually worked well, we'd not have outsourced call centers because center staff could actually be more productive. At least in that dimension. I've too many "account managers" at our telco who're authorized to do diddly squat to assist me, which is likely the reason senior mgmt don't care for which pbx / phones they use.

Outside of big "old economy" shops, however, this sort of thing could really help small dev shops get their customer relations under control. It's not organization per se - we can all be very controlled as to contact management - but take away just one bit of meatspace logic burden and i know i'd have considerably more time to write. Instead of contact management, i call it "distraction management".

I'm probably candidate for most boring post now, so i'll go curse at my phone's voicemail access system now!

</comment></comment></comment></comment><comment><USER8> Ive been a fan of www.kayako.com for some time, its got lots of features and is relatively inexpensive.

For a new product what I would like to see: Easy Template System Simple Back- end 1 Click Install (simple to wordpress) Live Messaging

1 area that I do not feel has been explored or taken advantage of is desktop software. I tried a while ago to create a desktop app that would sit in the system tray, receiving status updates and messages from a central system.

It would allow customers to stay up to date with system status without visiting a website and it could be used to notify of support replies, which should then reduce the "I didnt get a reply" issue.

</comment><comment><USER9> We have just set up tenderapp for our new Beta site www.pinkelstar.com (easy social network integration for iPhone and Android). I like it a lot so far. However, since we've just gone live I can't yet say anything about daily use. But the setup as pretty easy and convenient and the price was pretty decent too.

<comment><USER10> Seconding Tender. Beautiful and simple.

<comment><USER6> Maybe just my mass of plugins but FF3.6 isn't loading tenderapp, while chrome just fine. s'pose could be google's search redirect playing funny with requestpolicy, but am confused :~/

Apart from pricing, what's better than FogBugz, in terms of your decision? I can't find a straight features page, though straight to API faq so that's good.Seems to be Ruby, whilst Fog has XML and presumably, being .net a few other interfaces.

What confuses me is that OP wanted or hinted at a desire for a expanded feature set that Fog doesn't apparently have. Which are they that Tender gives? (I got the API point, that's good, but Fog being based on the CLR would give more ways to extend it if you took the site licensed version instead of the hosted version.

Also nitpicking, but Fog for the site install prices much more nicely than the on-demand version if you've >5 users.

I'm just utterly confused as to what was the OP's question was - it's so utterly vague - and led me to ramble off in different directions above. So what's the comparative benefit?

I'm a fan of Joel Spolsky's writing, and have evaluated Fog online, from POV of whether it would be useable by non programmer staff as comfortably as i'd like.

but i'm not yet a customer as you might guess from my above psts i've interest to do things which aren't standard for a code shop. Basically, i know the value i get is in much deeper planning and i'd have to write a lot of what's desired by my company, meanwhile Fog looks robust enough to use it as a way of pulling together a few things which aren;t totally out of its design intent. Outlook integration is a useful thing not so much for management but for syncing nicely with the cell phones we use, e.g.

I mean i'm not shilling for Fog, i just don't know what the criticism is except price. What is the OP disliking about the Fog APIs e.g.? If i put my cynical hat on, i'd just say this has been a "Ask /." kind of experience. I'm interested enough in the subject to engage, but have no way of really replying. Was the Q really "what's not Fogbugz out there are cheaper?" I genuinely don't know, nor can i (presently) find any links to older HN discussion which might be useful.

All this management software, whether for bug tracking or whether you make it front of house or anywhere in between is so useful to any helf - <USER2> business of any size, i seem to remember IBM paying a pretty penny to get ahold of Lotus who sussed the business pitch in the 90s.

I suppose that what _i'd like_ is a discussion as to what's state of the art, what companies or systems are really pushing the limits and getting things good and affordable, not a mere comparison with FogCreek which is very well written up for a company of any size.

I'll post separately with my confusion points, so please understand i'm probably just missing the point here.

best to all.

</comment></comment></comment><comment><USER6> Below is opinionated, but hopefully worth airing:

My view is that the field is over-populated.

Probelm is often definition of what's wanted. You end up with overlap between issue tracking and CRM, and if you add in a few outlier requirements you've described half the commercial software i see, from email installs to business automation, ERP and even EDI.

That's quite an exaggerated generalization of mine there, but the list of comercial and non-commercial software which can be put to use as you describe is enormous.

What are you really missing from FogBugz? I'm not clear why you're having to look elsewhere.

Mobile apps integration is simply not mature, or rather elegant, for any product i've seen. Zimbra, Lotus Sametime, come to mind as rather half hearted efforts requiring expensive additional licenses. Developer business sales groups really should wise up to the idea that a good mobile app is the thing small companies (mode 100 employees) i know want to have better systems, it's not a mere extra for new installs, it's the driver.

I forget when i first heard about "unified communications" but it wasn't even in the last decade. That promise seems to have been eternal. I feel we are inching closer, though, yet there's nothing breakout good in any category which i've encountered.

What i want is products with common rules engines, or something which can overlay that via apis. Smetimes i'd just like a regex to flag a string in an email as signifying a task is done and update a conversation thread. Sometimes i'd like a proper transaction written back to a sals DB. Sometimes i'd like to know my updates are delivered, with a message broker or transaction manager.It's applying consistency to that which makes me want a common interface to software as varied as voive, shceduling, management accounting (e.g. for that last one where there's a showstopper ticket, i want the revenue forecast to be dropped right away for that customer) Right now even trying for that flexibility requires some big bloat installs and a lot of work, some of it pretty low level.

As to scope for a new product, i don't think you can design a product out of the box for a large proportion of users in your situation. I believe ther attempt with XML and XSLT to find a common interface for data munging was a good try, but it turned out the only thing more verbose than the XML was the acronym dictionary. And a lot of XML i see is serializing ASN or whatever talks to the outside world of suppliers, basically a patch, not a consistent code base, so you are doing mapping schema for more than just your primary DB.

Sure, i'm being very opinionated / hand - waving. But maybe the best anser is to say you should think hard how important your support process is. Is it something which can really make you stand - out? If so, maybe you should invest in time and coding effort to get what you want. There really is no shortage of systems, apps and tools. Just nothing i see which can hook diverse office systems together. Example of this is getting desk phones to be useful. The best i've seen so far is the XMPP calls which SipExec and Aastra kit supports, but still you need some glue, and a lot of thought what you want delivered and why.

I'm emphasising voice in my post because no atter how i type, there are so many times a quick call really sorts out a problem in a fraction of the time. In my integration nirvana, i'd have the phone logs transcribed and appended automatically to the ticket. I'd have my desk phone call up the subject last discussed when i dial a number, preferably on the desk phone itself, so my concurrent work windows can be left alone.

If there's an opportunity to introduce a new product, i think the hurdle is the mind boggling variety of things any successful product would be wished to work with, and you'll bloat at the end of the day.

I doubt that what you want couldn't be done fairly simply on the data front. But UI on mobile devices is a good challenge to put it mildly.

Two favourite related moans of mine is the state of Nokia support, when they remain 50% of the smartphone game and are if anything still popular in the EU, for fresh sales, let alone installed base . . and the limits of active sync, which is often as not the best way to get email up for Nokia and Apple. I probably need to delve deeper into active sync to discover what really can be done with it, but i guess i'm looking at a oh so inspiring compatibility matrix if i use a non MS sync server.

If you want cheap during startup, mind, and don't have serious adversion to MS product, BizSpark is the best dev programme and you get a comprehensive license set for three years. Sure, it's no panacea, but i think their archittecture documentation is pretty intelligable and provided you want to hook in big name PBXs that would be a serious boon to my view.

Since a lot of what i do with contacts tickets and customer data is grunt style data monging, at least you can price a .net dev reliably. Check out LINQ, because it makes for handy glue if that's your environment. Some guys are doing or protoyping "stream" and "real time" BI with LINQ, certainly in some banks. It's not complete or fully mature by some views, but the project i'm looking at this year involved funnelling a lot of context data into priorities for contact follow ups, both externally and internally stimulated. I'd have gone hook line and sinker for this route already if MS would only talk straight as to whether their mobile handset voip integration will ever come to non MS handsets, which they keep hinting at but not delivering.

I do keep digressing, but i want pbx integration with screen pops available on a tab. Sybase Anywhere is designed to deal with this. OnRelay, a rare Brit company doing good things with SipExec has a partial solution. But it looks like a lot of work from where i'm standing, and my work lives or dies on customer inquiry handling with a big voice dependancy.To my rather simplified mind, i should not have to manually pass the context of a contact or resolution call all the time. This is specific to cell calls. Why does one have to juggle reading a screen, or finding an email thread, when to concentrate on actually speaking to someone, i need to defocus visually and actually pay full attention to what they're saying?

TL;DR version is there probably is a market for new product, but you've not exactly defined it, and for such a product to be truly useful it'd need to play nice with a big range of site installs, hardware (phones e.g. land and cell) and software, and it would be no use to me if the UI wasn't first rate.

final note: this is a very mature market in terms of contituent parts, but having so many failed products, so far as i see it (never yet seen a install at any price which approached comprehensive and simple interface or sub year install times) so you have some pretty big barriers to entry. Kind of project i'd love to do if i had substantial capital to bootstrap and a really caring dev / human factors team. Not something i'd like to do in a VC funded environment, because i am suspicious that product like these are not suitable for release early release often models, and that there could be a good deal of customer business domain specific knowlege required to be implemented first to make such a thing installable in a reasonable time. I'm still surprised how little use colleagues make of their high end cells, even though they're congnisant and if not geek, but very savvy, because somehow it never hangs together well when using under time pressure. Another hurdle is Nokia. You can cobble together anything Blackberry does with any recent Symbian phone, and you can write much more interesting things to that stack, but Nokia seem to go out of their way to hide the possibility. A chat i had with their PR reps in the UK was fascinating - no-one even knew of the capabilities, or that a big network operator in the UK thinks it's ok to blame the handset equipment for problems caused by their proxy servers, in writing no less . . . i figure if i attempted a well integrated system, i'd sure want it to work on that installed base as well as Apple.

Guess i feel strongly about this, so hope what i've said, if not actually helpful, is thought provoking in one way or another. I'd be interested in anyone whose encountered these problems generally and tried to write solutions, as there's ambition at my company to get serious about our systems in this aspect.

kind regards & good luck with your search!</comment>


How to Get Lucky in E-Commerce - <AUTHOR> http://foreverjobless.com/how-to-get-lucky-in-e-commerce/

<comment><USER2> On the buying sites thing, what is your opinion of flippa and other sites like that? Any tips on separating the dogs from ones that will actually "go"?

<comment><AUTHOR> Hey, I didn't write the post, but looking through the comments, he says he did get some sites from flippa. But mainly he did a "rediculous" amount of work.

Seems like that means he was spending all of his time looking for ecommerce sites anywhere he could think of – flippa most likely included.

So it's just a numbers game, he's plowing through lots of opportunities, finding some on flippa, finding some on other places... lots of hustling.

<comment><USER2> Sounds abOut right, thanks!</comment></comment></comment>


Tools to detect if your ISP is hijacking your DNS traffic - <AUTHOR> https://dnsdiag.org/ <comment><USER2> By the way, cool usage of markdown. Take a look at the page source. Amazing!</comment>


Ask HN: Does thirst in any way prove the existence of water*? - <AUTHOR>

*water being a place holder for any liquid that could quench thirst. <comment><USER2> "Proof" is strong word here, given that thirst is a subjective phenomenon and hard to pin down. But, at least along with the assumption that a large body of related knowledge to do with physiology, evolution and psychology, not to mention common sense, is broadly correct, actor reports of thirst seem at the very least to provide strongly indicative evidence for the existence of water.

Since the existence of water is not anyway in doubt, one wonders what the point of the question is. One analogous assertion sometimes made is that the craving for religious experience proves the existence of the divine, but I think this is a far shakier proposition, more like saying "thirst proves the existence of cherry cola."

<comment><AUTHOR> So you're in the desert desperately thirsty; you're fairly certain that you see water in the distance. That doesn't mean you'll get water -- you may die before getting there, it's a mirage -- but it's gotta mean that water exists for why else the driving, maddening, desire to drink?

Not talking about achieving, just talking about existing... remove all religious overtones (or undertones); just on a purely practical level -- if there was no such thing as water would you ever get thirsty?

<comment><USER2> I don't think we disagree that thirst presents strong grounds for belief in the existence of water, if we assume received knowledge about physiology, etc.

</comment></comment></comment><comment><USER3> I think you would first have to come up with some sort of concrete way of proving a lack of delusion. A delusional motivation doesn't have anything to say about reality.

It also apparently happens that people that are reasonably hydrated still think they are thirsty.

Another issue would be separating thirst from 'desire for unicorns'. Or whatever more reasonable example someone can come up with.

<comment><AUTHOR> Yes, I agree, being deluded that you are gonna get your thirst quenched and whether or not something that can quench thirst actually exists are two separate issues. Also, no matter how motivated you are to eat sand as a means of quenching thirst it'll avail nothing; on that we can both agree.

The second point is a missed placed / misinterpreted thirsting (your body needs something that it associates with drinking; electrolytes, sodium, etc) or an unfortunate inability of the body to process water. But that would seem to strengthen the necessity of water and not diminish it. In the former the powerfully felt desire to drink is so prominent that we confuse other desires for it, and the latter because the body will not stop desiring until satiated.

And the third seems to me to be just another way of saying that the desire is real but misplaced.

But really, I find it incredibly interesting that the conversation keeps wanting to be pulled into a discussion of aberrations of thirsting. Would you say that just because a minescule portion of the population has thirsting abnormalities of various kinds really invalidates the overwhelming majority of genuine thirsting?

<comment><USER3> If desires aren't consistently genuine, then they aren't useful as proof of anything, at least for the stronger meanings of proof (simply because any given momentary desire might be nonsense, which can't prove anything).

<comment><AUTHOR> Do you mean prove in the: > vulgar > scientific > logic > maths

Cause it seems to me they way I am using it and the way you are using it are different.

The first two are never provable in the sense of the second two.

<comment><USER3> I had sort of assumed you meant the second two.

(The question makes the most sense to me as a logic game...)

<comment><AUTHOR> When I created the question I added the phrase "in any way" to distinguish my mening the first two.

<comment><USER3> Sorry!

</comment></comment></comment></comment></comment></comment></comment><comment><USER4> It's a proof that it will eventually exist, because someone who wants it will invent it :)</comment>


7 Things You Didn't Know About Chatroulette's 17-Year-Old Russian Founder - <AUTHOR> http://www.sfgate.com/cgi-bin/article.cgi?f=/g/a/2010/05/10/businessinsider-7-things-you-didnt-know-about-chatroulettes-17-year-old-russian-founder-2010-5.DTL <comment><USER2> I kind of hate it for this kid, here he's got a great idea in his hands, and he (possibly) has no idea how much in the opposite direction this site will take in 5 years the minute he starts getting wined and dined by investors.</comment>


A Crypto Exchange CEO Dies–With the Only Key to $137M - <AUTHOR> https://www.wired.com/story/crypto-exchange-ceo-dies-holding-only-key/ <comment><USER2> [https://hn.algolia.com/?query=dies%20-explodes%20-Everest%20...](https://hn.algolia.com/?query=dies%20-explodes%20-Everest%20-Wright%20-Adams%20-Eating&sort=byDate&prefix=false&page=0&dateRange=pastWeek&type=story)</comment>


Wall Street Techs Take Secrets to Next Job at Their Peril - <AUTHOR> http://www.bloomberg.com/news/2014-07-17/wall-street-techs-take-secrets-to-next-job-at-their-peril.html

<comment><USER2> This is a disturbing trend by IBs flexing their legal might against their (former) software engineers. Before automated trading systems there were traders who carried their strategies around in their heads. On the surface it appears to be a game of scale because these strategies and systems are so costly if a trader walked off with a strategy it was one less bonus check they would have to pay. But if an engineer walks off with the design of a trading system and some strategies, then the firm might have to come up with some new strategies which is a costly affair indeed. Given the extreme nature of modern cybercrime laws to thwart terrorism and safeguard our economic infrastructure should they be used against ex-employees to safeguard intellectual property? Perhaps an interesting legal debate given the stakes.

</comment><comment><USER3> This is why no one wants to work at [Your Stupid Hedge Fund Here].

A while ago there was an article about how Wall Street was bemoaning the difficulty of recruiting top talent from tech.

Surprise: It isn't the money. There's plenty of money there. It's the total lack of understanding on how programmers create value. And this is the result.

There's so much to expound on here, it's truly amazing.

</comment><comment><USER4> "He continued to view information he wasn't allowed to see on his home computer, and repeatedly e-mailed confidential information to his personal Gmail account, including research on trading in China and a new method to improve models, the hedge fund claimed."

Wait, what? Did they wiretap him at home?

<comment><USER5> Most likely he was logged in remotely to work from home and they monitored his activity. They make it sound nefarious, but it sounds pretty much like a standard day working from home, except that he looked at some code he wasn't directly working on out of curiosity. I think that bringing criminal charges in a case of a minor violation of the employee handbook, as this appears to be, is ludicrous. But I guess it's supposed to send a message to all the tech geeks on Wall Street - don't get too uppity, and think twice about leaving for a better job if you might have ever violated the employee handbook.</comment></comment>


Show HN: documentation generator based on markdown files in node.js - <AUTHOR> http://beautifuldocs.com

<comment><USER2> DocumentUp creator here.

Cool project. Why can't we both collaborate on the same project instead and make it awesome? You seem to also be using CoffeeScript, eco, express and marked.

I have a few ideas about "Documentation as a Service", unfortunately I don't have enough time to devote to this project so it's kind of stagnating right now. Would appreciate your help.

So what made you create that?

<comment><AUTHOR> I made it because I like to write nice documentation for my projects but I was tired of docbook. I love markdown too :)

I thought of making a github webhook to automatically generate gh-pages using it but moved on another project in the meantime.

I like your idea of documentation as a service and I think you have something great with DocumentUp. We can definitively see if we can do something together. I'm deep in another project for the next two weeks (at least) but we can chat on gtalk (firstname.lastname@example.com - I'm in GMT+1) if you want, to see what we could do in the future.

</comment></comment><comment><USER3> Cool.

Looks very similar idea to <http://documentup.com/>, which in turn is similar to <http://readthedocs.org>

<comment><USER4> Yeah I thought the same thing only DocumentUp has a better design/concept.

</comment></comment><comment><USER5> This is a nice concept. If you are looking for a full-featured Markdown documentation product, I just released an app called GuideKit (<https://guidekit.com>). It's free for open source users. We track document versions with Git, offer a Markdown editor with live previewing, and every site has question & answer forums built-in.

<comment><AUTHOR> GuideKit seems really nice. Nice UI. However, one of the reason behind beautiful docs is to keep the documentation as files. That way, everything is in git and readable from the shell/github too.

If you add an "import/sync with github" option, this would be really great.

<comment><USER5> Funny you mentioned importing from GitHub - that feature is mostly complete and should be launching sometime this week or next. I understand where you are coming from about keeping everything in files, I think that is perfect for technical users.

I masked some of those details in GuideKit to hopefully attract both technical and non-technical users (like support teams that don't know how to use git). But, I plan to offer pull/push capability in the future so you can edit your GuideKit docs locally with a text editor and push them back up.

Anyhow, nice job with beautifuldocs. I applaud any tools that help/encourage developers to write nice docs!

</comment></comment></comment><comment><USER6> If you just want to preview your Github Markdown files (e.g. your README.md) locally before committing them into Github, check out my Node.js-based Github Flavored Markdown Server: <https://github.com/ypocat/gfms> (it uses Github's CSS and properly parses the "```" code sections).

</comment><comment><USER7> It would be interesting to do WYSIWYG editing of the Markdown docs here with <https://github.com/bergie/createMarkdown>

<comment><USER8> If by 'here' you mean HN comments and text bodies, I'd support that 100%. While many would go over-board, I think that the communication quota would increase given better looking communication.

<comment><USER7> Nope, meant the documentation tool this thread is about :-) Though agreed that more comprehensive formatting might not hurt HN either</comment></comment></comment>


OCaml 4.07.0 has been released - <AUTHOR> http://ocaml.org/releases/4.07.0.html <comment><AUTHOR> When there are new releases of OCaml, usually the first comment is about multicore and modular implicit. They are not yet there, but things will start changing soon. Refer to [0, 1] for more information.

[0]: _Ocaml-multicore: report on a June 2018 development meeting in Paris_ [https://discuss.ocaml.org/t/ocaml-multicore-report-on-a- june...](https://discuss.ocaml.org/t/ocaml-multicore-report-on-a- june-2018-development-meeting-in-paris/2202)

[1]: _Modular implicits_ [https://discuss.ocaml.org/t/modular- implicits/144](https://discuss.ocaml.org/t/modular-implicits/144)</comment>


Ask HN: How to make this Net Neutrality simulation better? - <AUTHOR> I made a net neutrality simulation - what could make this better and/or more accurate?
I just left the hair salon and of the 15+ people in the salon only one had even heard of Net Neutrality (I live in San Francisco)!
I want to create this simulation so people can experience what is at stake.
Thank You
http://net.codetocanvas.com/ <comment><USER2> EME is supported by the same companies that support net neutrality. Is there something amiss? Hypocrites much?

</comment><comment><AUTHOR> I am going to be making updates to this all day. Any suggestions are appreciated.</comment>


A Practical Computer Science Primer (2008) - <AUTHOR> https://learn-cs.pbworks.com/w/page/15774601/FrontPage

<comment><USER3> That seems like a fairly complete syllabus, quite like what I spent 4 years learning in school (2 undergrad, 2 post-grad), minus the internships and actual work (and the mobile and web apps that we didn't have back in the day...). I wonder if these courses really took place and how they turned out.

Some minor topics missing are things like compression, multi-threading and locking, etc, but I suppose those are oversights, or there's just not enough time.

I think one glaring omission is the UI/UX. It's the last topic under "graphics," and I really wonder if they'll reach those last topics. I think usability, windowing, design, typography (as seen today on HN), and new interfaces (touch screen, Kinect, etc.) could be a topics for an entire lesson of its own. If only because I expect people who take such a course to be more practically oriented (as in teach myself to program so I can develop my app idea).

Also, given such an audience, I think an extra lesson covering real-world examples would be very helpful. The lesson on web apps and mobile probably covers a lot of the end-user cases, but I'm thinking about industrial applications, modern airplanes, big installations such as banking or stock exchanges, a mainframe that's still running somewhere, drawing-board to release of a Pixar movie, a data center or just a colocation facility (field trip!). And it might make sense to put this lesson as an intro, where even if the students don't understand the details at first, the rest of the course can be mapped to real-world examples.

But I do wonder about the OP's final intent of compressing years of learning into weeks of lessons. I'm not sure people can retain and make good use of so much information without putting it into practice. I would love such a course for some other topic (civil engineering, political science, etc.) but it would be more about expanding my general knowledge, much like I read wikipedia for fun, than about being able to do something with it.

</comment><comment><USER4> OP here, some 5+ years after having authored that content and taught the class. Astonishing that this got surfaced - I was very surprised to receive a flurry of inbound requests to join a class I ran half a decade ago. I'd love to know how best to package the material - what would folks like to know? And how would folks like to learn it? Where are existing systems like Khan Academy and Codecademy falling short?

<comment><AUTHOR> Right now it seems there's a great interest in building mobile apps, even if they are just web based. Ideally, I'd like to use the time commuting to learn on a mobile device or tablet. This is exactly where existing systems like Khan Academy & Codeacademy are falling short, but they are still really excellent systems. Thanks for teaching those classes, I went to a few of them and learned lots.

<comment><USER4> Thanks, barce. I'm curious - by "using time communiting to learn" does that imply that a podcast would be the right form factor?

<comment><AUTHOR> Yes, I think a podcast would work for commuters who are coders - <USER2> the right form factor.

</comment></comment></comment></comment><comment><USER5> Neal Stephenson novels required reading eh?

</comment><comment><USER6> That's one heavy duty course! Looks like it covers almost all of my 4 years of undergrad.

</comment><comment><USER7> I wish there was something like this near me, I'd do it in a heartbeat</comment>