Show HN: Codemap – Codebase Visualizer for JavaScript, TypeScript, and Python - <AUTHOR> https://codemap.app <comment><AUTHOR> Hey HN, I'm the creator of Codemap.

Codemap is a codebase visualizer that displays the structure of function calls of any Javascript, Typescript, or Python code. Given a local repo, Codemap statically parses the code and renders a directed graph of all function calls in the least cluttered layout. It helps programmers familiarize with a new codebase, trace possible scenarios that lead to a bug in a function, or understand the scope of impact when making a code change. Codemap runs offline on your local machine and never sends any sensitive code data to remote servers.

I built Codemap because I noticed that good engineers always spend quality time understanding the code architecture before making changes. This process is crucial, and it usually requires drawing a function call diagram on a whiteboard or in the head. Some people use IDE "Find usage of function..." search box for a million times, some people ask senior engineers for tribal knowledge, and some just get lazy and start writing code with ignorance.

As a software engineer, I think there's a better solution than this tedious and error-prone practice. I looked at existing code visualization tools and felt that the visualization tends to be overly cluttered and not that user- friendly. I believed that, with some level of design, I can build a tool that is pleasant to use, easy to navigate, and versatile for any codebase in the supported languages.

During the development of Codemap, I would sometimes grab a popular Github repo to test the Codemap app, and I often find myself exploring its call graph for hours to satisfy my genuine curiosity of "How does <popular project> work?" I'd encourage curious HN readers to visit the website and see screenshots of call graphs from React, Keras, Django, and Typescript (yep the one from Microsoft).

I would love to hear what you love and hate, as well as any questions you may have. I'll be here to respond the best I can (PST time). Thank you :)

<comment><USER2> Hey, great stuff, but my question is whether produced visualization/graph is similar to how would you "see" the code in your head yourself?

The problem of familiarizing/navigating textual codebase is embarrassingly unsolved, and I was exploring this in the post "Rethinking visual programming with Go" [1], but yet my main issue with visualization tools is that they offer intermediate representation of the code, which still has to be "decoded" into proper mental map.

I see the future of these visualizaton approaches in AR/VR. Code visualization should match mental map visualization as much as possible, and the better we can use spatial tools for representation the better the result.

[1] [https://divan.dev/posts/visual_programming_go/](https://divan.dev/posts/visual_programming_go/)

<comment> amw-zero I think a graph is how we perceive code, I just think the human brain has a very unique way of comprehending graphs. It seems to me like the graph is "indexed" in multiple different ways, so that multiple paths exist between nodes. The brain is also better at categorizing nodes - e.g. "fetch everything that I know about colors in general."

I definitely think of code in graphs, but it's definitely beyond just a static graph diagram.

<comment><USER3> Graphs and tags for me, It's hard to describe but I can picture parts of the codebase I've worked with but only roughly until I need to interact with them then I can hold those details in my head but I lose that clarity somewhere else - I think it's a cognitive limit I have, I don't know where that lands against other developers and to an extent I think it can be trained.

As a lead/senior the thing I've observed as the crucial difference between say someone with a year or twos experience vs someone at my point is how fast I can get up to speed with a codebase I've never seen/touched.

Partly I think it's knowing how to use the tools better to get the answers I need and partly pure cyniscism, I've learnt to turn off the "if this was done sensibly" bit in my head so I can actually see what it's doing over what it probably should have actually been doing.

One of the things I really focus on with my juniors is how to use the tools available (and an A4 pad and pen) to figure out what the system is _actually_ doing.

<comment><AUTHOR> Can't speak for you, but I think a lot of that comes from experience, e.g. put out fires caused by whatever stupid reason, and eventually build up awareness of what could go wrong. Therefore, without reading too much into the code, simply being aware of their existence becomes largely helpful.

</comment></comment></comment></comment></comment><comment><USER4> This looks cool! Pricing looks reasonable.

My main piece of feedback is that it's still a bit difficult to tell what this does, and more specifically how it solves the problem of understanding a codebase. For example, how well does it understand my code (does it use language servers?), are there any views other than the tree, can I jump into a file and choose a function, etc

Even if I download it to try it out, the 100 function limit on the free plan seems like it would be quite limiting to trying the tool as part of my workflow (I think a codebase that I have difficulty navigating tends to be larger than that).

The tool is cheap enough that I wouldn't mind skipping the free plan entirely, but it doesn't seem like there's a way to purchase just one month.

One question about the performance side, is the app native, and how massive of a codebase can it handle while staying snappy?

<comment><AUTHOR> Thanks for the feedback and great questions! Let me address your questions below.

\- You nailed it, Codemap uses language servers (Typescript for TS/JS, Jedi for Python) to understand the code.

\- Tree view is currently the only layout that's available. If you have any thoughts on this please let me know!

\- It doesn't support jumping into a file a choose a function, but it supports a few interactions that may be as useful. A few examples are: 1\. zoom in/out, either horizontally or vertically 2\. limit render scope by folder and files (useful to keep performance up and avoid cluttering) 3\. showing only nodes that are upstream or downstream of a particular function (useful for tracing the scope of impact) 4\. quick code preview for node (function definition) or edge (function call)

\- You're right, the 100 function limit is a little harsh. The goal is for people to try out the tool with a few files and get a taste of it. The price is pretty affordable though, as I want Codemap to serve the global programmers.

\- There's currently no option to purchase just one month, but I'd definitely consider it!

\- This is an Electron app (please don't scream :p) with the goal of market verification and gathering early feedbacks. If this app solicits enough interest, I'll seriously consider re-building a native app. That said, from my experience Codemap remains snappy up to 2000 functions that are moderately interconnected, at which point the issue of cluttering outweighs its slowness, and limiting the rendering scope (folders/files) becomes necessary.

Again, thanks for the thorough feedback! Let me know if you have more questions.

</comment></comment><comment><USER5> Some time ago I tried building something like this too, but lost interest at the end. Feel free to pick ideas if you find them interesting:

[http://iswaac.dev/blog/2019-12-14/](http://iswaac.dev/blog/2019-12-14/)

The basic idea is to use nested boxes to represent ownership and lines to represent dependencies. I only did the first part to some extent: parsed JS and rendered the AST into a treemap using WebGL.

What I actually envisioned in my mind is a 3d nested boxes rendered in a "galaxy style" where you can fly from one box to another, click on boxes to show their sub-boxes and see the lines that represent connections. The 3d space would allow to unclutter the image.

Edit1. While I think that nested boxes is the natural way to represent ownership, it's often difficult to figure this ownership from the code. For example, class A is really the owner of B and C, but those are given to A as args in the constructor, so to an unsophisticated analyzer tool, B and C would be just deps. Perhaps the tool should allow the user to adjust the relationship status between objects based on some a priori knowledge. The changes would need to be saved, so next time the diagram is rebuilt from code, the visualizer would remember that A owns B and C. I believe this would be really useful in big companies that struggle to keep such diagrams up to date.

Edit2. I'm reading my own note about WebGL and matrix projections and laughing: I wouldn't write that nonsense with my current understanding of webgl.

<comment><AUTHOR> Awesome project! I particularly love the AssemblyScript visualization resembling a city view. Thanks for sharing it! I'm not sure I follow the definition of ownership, and maybe you can help me out here. Does ownership mean that, if class A owns property B and C, iswaac renders box A enclosing box B and C? How does it apply to function relationship when it's not a strictly nested tree? This reminds me of Firefox 3D view [1] of HTML DOM elements, which is another handy visualization that's unfortunately discontinued. I think it helps frontend engineers fighting against spaghetti HTML/CSS code.

[1] [https://developer.mozilla.org/en- US/docs/Tools/3D_View](https://developer.mozilla.org/en-US/docs/Tools/3D_View)

<comment><USER5> It's the usual "A contains B" relationship. At the top level, the entire project contains everything else, so there is one big box with many boxes inside. Usually, the next level is folders, then files, then classes and methods. But at the classes level, it gets tricky. There's syntax hierarchy, when class A contains method B, and there's logical hierarchy, when we know that in a working app, class A is the owner of class B, meaning that an instance of A creates, manages the state and destroys an instance of B. The visualizer needs to somehow account for this or let the user specify this relationship.

I think that a 3d webgl visualizer would be cool, but a graphviz style diagram generator would make money.

</comment></comment></comment><comment><USER6> This looks cool. Indexing node_modules feels like the wrong default. I waited several minutes with cpu at 100 after startup for a fairly small React project.

And maybe the free tree could start at a more useful place? index.js or App.jsx or App.vue? In my case, the 100 functions are all instrumentation and polyfills, which doesn't give me much sense of the potential utility.

In a Django project, I see...nothing.

Good luck! This seems like it could be useful.

<comment><AUTHOR> Thanks, points well taken! I think parsing .gitignore plus some basic exclusion like node_modules would be a good start. For the free version, you're totally right that it should pick 100 functions that are more interesting to play with. To avoid assuming the project structure like App.vue, I'll probably prioritize showing connected components with the most nodes. Sorry for the issue on Django projects, I'll test it out and see what's going on there.

</comment></comment><comment><USER7> How do you determine whether python is installed?

I'm getting this error on OSX: > Please make sure Python3 is installed on your computer.

My python3 installation works fine and is located at: /usr/local/bin/python3

<comment><AUTHOR> It checks whether `python3 -V` ends successfully in a Node JS child process. Is it possible that `python3` is not available in your environment? Or perhaps `python3` isn't linked properly? If you use brew, `brew link python3` could be helpful.

</comment></comment><comment><USER8> I've been using SourceTrail[0] since they open sourced it to get my head around a large Java project. It doesn't yet support TypeScript projects so I'm really interested to see how Codemap goes on our React app.

[0] [https://www.sourcetrail.com/](https://www.sourcetrail.com/)

<comment><AUTHOR> Yup, SourceTrail is probably my favorite before deciding to build Codemap. Since my day job mostly use Typescript, I have to build this to serve my own need.

</comment></comment><comment><USER9> I'm unable to get anything other than a message saying "Empty graph" for Python. Does it need to _know_ about venv somehow?

JS works fine.

<comment><AUTHOR> It should work regardless of venv. Could you shoot me an email (in my profile) so I can follow up with you?

</comment></comment><comment><USER10> ``` Uncaught (in promise) abort("Cannot enlarge memory arrays. Either (1) compile with -s TOTAL_MEMORY=X with X higher than the current value 16777216, (2) compile with -s ALLOW_MEMORY_GROWTH=1 which allows increasing the size at runtime but prevents some optimizations, (3) set Module.TOTAL_MEMORY to a higher value before the program runs, or (4) if you want malloc to return NULL (0) instead of this abort, compile with -s ABORTING_MALLOC=0 "). Build with -s ASSERTIONS=1 for more info. ```

<comment><AUTHOR> Sorry about the issue. It seems to be struggling with memory while rendering the graph. Are you rendering a large graph? Could you try a smaller scope by restricting the number of files and folders?

</comment></comment><comment><USER11> I have done this kinda thing to great effect using Doxygen, although this solution adds a bit of dynamism that is really appreciated. Having the call graphs, especially for a new/unfamiliar code base is a great way to gain understanding. Kuddos on the launch, i think a lot of engineers will find this pretty useful once you expand on it a bit.

<comment><AUTHOR> I really appreciate your kind words! From your experience using a static graph by Doxygen, is there any feature in particular you'd love to have? I'm actively improving this tool based on the awesome feedback from HN, and what you wish for will have a large chance of landing on the next upgrade :)

</comment></comment><comment><USER12> OMG this would have been handy so much. I used to track the call stack by following functions and writing it out to a text file.

<comment><AUTHOR> Thanks, I have the exact same habits until this :)</comment></comment>


Clickbait Snail Mail: How companies try to get me to open the mail they send - <AUTHOR> https://alexanderell.is/posts/clickbait-snail-mail/ <comment><USER2> The discounts these companies get from the postal service for mailing in bulk, presorting, using specific labeling, etc can get pretty large. Just a couple cents a mail piece.</comment>


What Engineers Want – A Rebuttal to the 501 Manifesto - <AUTHOR> http://blog.herlein.com/2012/04/what-engineers-want-a-rebuttal-to-the-501-manifesto/

<comment><USER2> Good points. Makers need to make and use their creativity ( <http://spin.atomicobject.com/2012/04/08/do-what-you-love/> ). We cannot just be butts in seats. The 501 guys seem to want to just be a cog in a machine, but that is not enough for me. I think other makers would agree.</comment>


Ask HN: How do I find hacker news link to already submitted story? - <AUTHOR> "Snapchat Is Buying Bitstrips" was just posted. But I can't find the article. I tried: "site:news.ycombinator.com snapchat is buying bitstrips" on google.  Nothing. Is there a better search? <comment><USER2> There's a search bar at the bottom of the home page on hacker news?...

<comment><AUTHOR> ha! i'm blind, thank u

</comment></comment><comment><USER3> search box at the bottom of the page? Misses stuff that was _just_ posted sometimes, because it takes a minute or so to show up, but otherwise works well.</comment>


Poo found on every McDonald's touchscreen tested - <AUTHOR> https://metro.co.uk/2018/11/28/poo-found-on-every-mcdonalds-touchscreen-tested-8178486/ <comment><AUTHOR> Just as a reminder that every technological revolution brings its own unique risks with it. I'd be surprised if this was limited to McDonald's, too.

<comment><USER2> Currency is also often tainted with bacteria.</comment></comment>


Cappuccino meetup/party on Monday, SF, hosted by 280 North and Sofa - <AUTHOR>

280 North is co-hosting a small event with Sofa (http://madebysofa.com/ - they're responsible for Cappuccino's new theme, "Aristo") on Monday for Cappuccino developers and anyone else who wants to join us (if you're reading this on HN you're invited).
http://cpcoder.eventbrite.com/
7-9PM, Martini Room at Jillian's in Metreon across the street from WWDC (Moscone West).
Also, Francisco will be speaking at CocoaHeads on Wednesday at the SF Apple Store: http://theocacao.com/document.page/606 It's open to the public but get there a little early as it will fill up fast.
We've got some competition from dozens of other WWDC parties (http://blog.quazie.net/2009/05/wwdc-partiesevents/) including Loopt :) <comment><USER2> correction: <http://madebysofa.com>

<comment><AUTHOR> thanks, fixed.</comment></comment>


Major Apple Product Announcement Set for Wednesday, Jan. 27 - <AUTHOR> http://digitaldaily.allthingsd.com/20100104/major-apple-product-announcement/ <comment><USER3> Can anyone confirm that this is actually going to happen? I keep seeing 'rumors confirmed!' but no one has shown an invitation or any actual confirmation from Apple. For all we know they may be reserving that convention center for a company retreat!

<comment><USER4> It's not an official confirmation, but FWIW the WSJ is reporting that Apple "will unveil a new multimedia tablet device later this month", citing "people briefed by [Apple]" - <USER2> [http://online.wsj.com/article/SB1000142405274870358090457463...](http://online.wsj.com/article/SB10001424052748703580904574638630584151614.html)

</comment></comment><comment><USER5> I guess I'm alone in hoping for a Nehalem-based Macbook Pro revision (I really want a 17" WUXGA MBP with an i7 CPU to replace a MBP 3,1 15")

<comment><USER6> There will be new MacBook Pros at some point in the near future – but I think Apple is less and less inclined to give any such "boring" changes their own events.

I think Apple has been moving away from situations where they are forced to present something – anything – (like Macworld) and will try to only use events for really new stuff. The two lone exceptions are the WWDC and the yearly iPod event. You might see simple spec changes in the WWDC keynote (probably only mentioned in passing), but I don't think you will see them anywhere else.

(Design changes might be something they would present at such an event. But since we just lived through the change to the Unibody design I don't think we will see changes in the next one to two years. That would be unnecessary and stupid.)

</comment></comment><comment><USER7> If they're announcing the tablet in January, I wonder what kind of release date they're looking at. Does anyone know what the average announce to release cycle is for Apple products?

<comment><USER8> The original iPhone timeline was a January announcement with an end of June release.

<comment><USER9> Although, the iPhone had significant external schedule factors which pushed an announcement: FCC, AT&T, etc. Unless this device can connect to a cell network, I don't see why they wouldn't pull one of their patented "And it is available.... right now!" things.

<comment><USER10> I doubt it; content and software would not be ready in time.

The iPhone SDK had a six-month public seed time for devs to create their apps for the opening of the iTunes app store, but there's been no public SDK for the tablet yet. It'll be announced now, out in June.

<comment><USER11> The iPhone wasn't released with an SDK or App store though. While I think it would be unlikely for Apple to launch a tablet / new mobile product that doesn't have an app store, it wouldn't be the first time.

</comment></comment></comment></comment></comment><comment><USER12> Intel just released new 32nm laptop CPUs with embedded GPU, so there'll probably be some new laptops with those chips.</comment>


How I Got 2200 Pre-Signups for My SaaS in 2 Weeks, Then Failed Hard - <AUTHOR> https://mattkremer.com/how-i-got-2200-pre-signups-for-my-saas-in-2-weeks-then-failed-hard/ <comment><USER2> Actually, those stats are not horrible.

If you dig around online (sorry, can't find the link right now), you'll find people reporting conversion numbers for SaaS free trials --> paid accounts at:

      10% convert to a free account
      15% of free accounts convert to a monthly subscription
      </comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>
      0.015% overal conversion rate

To put it loosely in perspective with your numbers:

      2,200 wait list signups
      9 people signed up and used it
      </comment><comment></comment><comment></comment><comment></comment><comment></comment><comment>
      0.004% conversion (a quarter of the way there)

I didn't read your follow up, but my first thought would be to carve out a hyper targeted sub-niche of users.

For example, programmers who work at a < 10 person startup, located in San Francisco, who can be found listed on Twitter/Facebook/Etc..., have purchased monthly subscriptions for software tools in the past and wear yellow t-shirts :)

Market to them with a very personalized message (easy since it's a very targeted sub-niche) and see if you can get closer to that 0.015% mark with that targeted messaging.

But you may already have done this or something similar detailed in your follow up post :)

Just my two cents...

<comment><USER3> Hey coffee, thank you very much for your insights! It's appreciated :)

I actually wound up converting 4% of the 2200 to one-time paid accounts, and am experiencing a lot higher use with the new version. 20% of visits result in 2 or more people collaborating on a file.

Sadly I'm struggling to get actual monthly/yearly subscriptions, however I think this is more of a CTA problem than a problem with the product. I really need to focus on getting the landing page better, and having a better CTA within the app.

Thanks again!

<comment><USER2> Yea those sound like good results.

It would be interesting to see, over say 1,000 free trials, how many you could covert into monthly subscriptions and understand if the numbers I mention align or not.

Sounds like you may do much better. Would love to hear the results when/if you figure it out.

</comment></comment></comment><comment><USER4> That's not a failure to me, Matt. Giving up on it was, IMHO.

You don't just build something and expect people to use it. You have to get feedback from them, listen carefully and adjust your product to what the users really want. You did a splendid job on getting that initial traction and people signing up, now figure out from the users why they decided not to use it.

Keep it up, man. People want it and you have proof :)

<comment><USER3> I completely agree goshx! This article was actually from a few months ago, I've since relaunched Kobra and it's doing well! Almost 700 users, over 5000 files collaborated on and my initial launch sale netted me $2100:

[https://mattkremer.com/how-i-made-2033-in-4-days-while- valid...](https://mattkremer.com/how-i-made-2033-in-4-days-while-validating- my-side-project/)

<comment><USER5> Nice work, congrats!</comment></comment></comment>


Tracking Changes using JavaScript - <AUTHOR> http://open.blogs.nytimes.com/2012/01/23/introducing-ice-writing-for-the-web-first/

<comment><USER2> love that someone in news/publishing is pushing crazy journalists to write for the web ... been doing this on content/copy for years using what? Google Wave

<comment><USER2> See also: Etherpad, springs to mind</comment></comment>


Tom Knight, Godfather Of Synthetic Biology, On How To Learn Something New - <AUTHOR> http://www.fastcompany.com/3000760/tom-knight-godfather-synthetic-biology-how-learn-something-new <comment><USER2> Save yourself the time: " I like to read books, three or four at a time. I rarely read books all the way through. I'll get a few books on a subject--you want single-author books, someone with a well-defined point of view--and read a section, and then switch to a different book and read about the same thing. I keep switching back and forth--it's a great technique because you get to look at the same subject from many people's perspective. That turns out to be actually really useful."

<comment><USER3> I thought the whole article was pretty interesting. I participated on an iGEM team a few years ago under a professor whose story was pretty similar to Knight's.

It's refreshing to see established academics willing to take a sophomore level biology course so they can learn the fundamentals of how a biology lab works.</comment></comment>


How I Made my Blog 2.3x Faster - <AUTHOR> http://blog.alexbrowne.info/how-i-made-my-blog-faster/

<comment><USER2> "Ruby/Rails" <-Theres the problem

I know HN loves these frameworks, but seriously, the JVM is waaay faster... Too bad there wasn't a dead simple JVM or Java based blog engine.

<comment><USER3> Did you catch that afterwards he still wasn't using the JVM?

Any time you're serving up gzipped static pages instead of loading _any_ language runtime, JVM or not, you're going to see huge speedups.

<comment><USER2> My apologies for "implying" that. I facepalmed when I saw, "slow" and "ruby/rails". So I digressed into a rant about the lack of a simple blog engine for Java, which would likely run much faster than a Rails solution. I'm fully aware his solution is to just pushed static files, and I even bet he's having Apache do linux sendfile, which would pretty much idle the CPU.

Better?

<comment><USER4> What do you find lacking in the countless existing blog engines for the JVM? It seems to me that hosting blogs on the JVM is about as well-supported as hosting them with Ruby.

</comment></comment></comment></comment><comment><USER5> Everybody says it's the bees' knees, but Ruby on Rails is terrible. I speak from experience, I administer non-public installations of Redmine and Gitlab.

First of all, the language syntax is awful and incomprehensible.

Then there's the Byzantine deployment process (although, to be fair, it seems endemic to modern web development; it's just more obvious in Ruby since most Ruby programs are web apps).

Finally, the startup for a Ruby application is _incredibly slow_ \-- worse than Java ever was, even in the Bad Old Days running 1.2 on a 300-ish MHz Pentium II (which was a state-of-the-art desktop at the time).

Hint: If someone makes a post to the Github issue tracker for your web application saying "Increase timeout from 30 seconds to 300 seconds" [1], a one-line patch changing a number from 30 to 300 does _not_ count as solving the problem. There is _no reason_ that a web app should _ever_ have an expected load time greater than 30 seconds.

[1] <https://github.com/gitlabhq/gitlabhq/issues/694>

</comment><comment><USER6> While I personally like Jekyll, and the added simplicity when it comes to hosting, there should be almost no performance difference between a "dynamically" generated blog based on Rails/Sinatra behind Cloudflare and static hosting on S3.

The only user generated content on a blog are usually comments. With just a bit of added caching headers, I'd say that 99+% of all page hits on a blog can be served out of Varnish. You can even use rack-cache if you don't mind the performance penalty. If you're ok with using e.g. Disqus (as Octopress does) for comments, you can serve 100% of them and just purge the cache when adding new content. This would probably also work when doing a purge for new comments. This way, you have to do the dynamic computation exactly once per page and you're done.

Adding the CDN properties of cloudfront to S3 is nice, but Cloudflare seems to provide a very similar solution for "regular" sites.

I'd say this is similar to comparing AOT compilation (Jekyll) vs JIT (dynamic generation + caching).

<comment><USER7> I converted my blog[1] to be static this past weekend using all AWS (S3+CF+Route53) and it went relatively painless. But I decided to use middleman instead, and after using that I don't think I'll be going back to Jekyll.

I still have a Github.com pages account that uses Jekyll because its free, but since its so damn cheap I may convert this to AWS stack as well.

The largest problem I was running into was permissions and invalidating the CF CDN. I am thinking of rolling a simple Markdown editor using node-webkit and scripting up some things to automate everything.

[1]: <http://thoughtlessbanter.com>

<comment><USER8> I wrote a deployment script for s3/cf/r53 in python last week. It's pretty janky right now and is very oppinionated on a lot of things, but it handles invalidation requests for you and the code is simple enough to easily modify to your liking.

<http://www.github.com/heyimalex/s3tup>

</comment></comment></comment><comment><USER9> I don't keep up on what various platforms offer but wasn't the original movabletype a static site generator? You entered a new post, clicked generate, it made a bunch of pages. It was only later that they added a dynamic option.

Blogger was static too. It was just a cloud based DB. You gave it an FTP name/pass and it logged into your site and uploaded the new files.

</comment><comment><USER10> Web 3.0 - static content? ;)

<comment><USER11> That or it's anti-thesis of sorts, i.e. websites that are rendered almost entirely on the client side.

<comment><USER12> Websites are already rendered entirely client side.

<comment><USER10> I think he is referring to having the clients download a copy of the database and a big chunk of javascript to create views and forms to use the database. Syncing the database with the server at specified intervals, etc.

Maybe that can be Web 4.0 - Any ideas for Web 5.0? ;)

</comment></comment></comment></comment><comment><USER13> I recently wrote a blog post about how Rack::Cache and ETag work [http://blog.craz8.com/articles/2012/12/19/rack-cache-and- eta...](http://blog.craz8.com/articles/2012/12/19/rack-cache-and-etags-for- even-faster-rails), which is particularly important to know for Heroku based apps.

The key part is that, for public content (and blog posts tend to be public), the Rack Cache can be used to serve these pages directly from the cache store (usually Memcache) with minor database traffic needed, _even for people who have never seen this content_.

That last part was the surprise for me - surely ETags are only used by return visitors! Rack Cache makes ETags work for new visitors too.

I think I can get my blog to run almost as fast as a static site, and I'm working towards getting that done and documenting it as I go.

(In addition, Heroku seems to be adding Varnish headers to my responses. They say they don't use Varnish in Cedar apps, but this is clearly not correct)

</comment><comment><USER14> I was looking to move my Drupal site to Jekyll but quickly got bored. I therefore moved the site to Sinatra and now Rails, yes it's slower than jekyll and the rails site is slower than the sinatra one but more fun.

<comment><USER15> re: "more fun". Yes, having a faster site that uses less resources is great, and very probably important for those seeking audiences as large as possible. But coding my own CMS and making it just like I want it (minus where my abilities limit me in that) is something I never would go back on. It's simply so much more fun than, say, just using wordpress (not to mention something even more basic).

Sure, my site is just "my stuff", quotes and links I collect, maybe a rambling here and there. Therefore my comment is not quite on-topic, it's not the same kind/league of site. But I just can't find anything exciting in pure speed, I want something that I like when I look at it and use it, not something I like because I know it's very popular. I like having variable and filterable views on content, user preferences, etc. too much.

I don't want to pretend I'm some kind of artist; I'm a shoddy coder, and not a great designer either. My CMS is unusable for anyone except me I'm sure. But still I take pride and enjoyment in whatever it is I'm doing, and at the very least I would encourage everyone to make something that is about ideas and features more than performance, even if as a secondary/private site. Minimalism is sometimes overrated. It may be more effective when dealing with many people, but it's also a bit sterile. So maybe do both, one for business, the other for inspiration.

<comment><USER16> Agree. My first reaction when reading the article was "your blog was massively overengineered". And while it was (after all, it got replaced with static HTML pages), I'm sure it was fun measuring and tweaking the performance and learning about building a scalable infrastructure.

</comment></comment></comment><comment><USER17> Isn't it obvious that static HTML is served faster than dynamically generated HTML? What's so new about that? Please let me know if I am missing something.

<comment><USER18> It's actual data from one actual (and very common) use case.

Also, the document itself is just one fairly small piece of the whole puzzle.

</comment></comment><comment><USER19> Ummm... not to be the wet blanket, but what business advantage is there to your blog loading 2.3x (or even 5x) faster? I understand you may occasionally be adding content that helps plug your work, but in general I am less concerned about the speed of a long article loading than I am the content being worth the much longer time it will take to read.

Maybe it's just me?

<comment><AUTHOR> There's not really a tangible business advantage. Sure, I put a brief plug in some of my posts, but that's not really the point. For me the whole point was just to explore new technology and push it as fast as it can go. I learned a lot from the process, I found it personally intriguing, and maybe I can even use some of what I learned in future business ventures. Other people might be able to take away something from it as well, however small.

(I know it's a pretty hefty post, hence the tl;dr at the top. No one is forcing you or even asking you to read the whole thing)

</comment></comment><comment><USER20> Wow. I'm actually impressed by the load speed of that page.

As someone who lives in South America I've gotten used to sites loading in 2+ seconds, but this one feels almost instantaneous.

</comment><comment><USER21> I'm sure this was a fun exercise but I see this just as a limitation of heroku. Normally you could've switched to full page caching in your rails app and get the same results.

<comment><AUTHOR> True, but there's still an advantage to using a cdn in terms of worldwide performance.

</comment></comment><comment><USER22> From previous HN threads on Static Site Generators I have seen people recommend Punch and Nanoc. How does those compare with Jekyll?

<comment><USER23> Jekyll is slow as molasses on anything over the trivial level, I haven't used Punch or Nanoc so I can't comment on those.

</comment></comment><comment><USER24> I guess the price difference between the two setups is huge. I'd love to hear more about that.

<comment><AUTHOR> Actually the price will end up being pretty similar. It was more or less free when I was hosting through Heroku (pennies a month, and that was just for images on CF). It's hard to predict exactly (depends on traffic) but I'd be surprised if it costs me more than $1 per month with the new setup. Will find out for sure at the end of the billing cycle.

<comment><AUTHOR> Update: For the month of December, Amazon charged me $1.06 for S3 and CloudFront combined. This includes the ~million requests I performed during benchmarking.</comment></comment></comment>


"(null)" is a bad idea - <AUTHOR> http://www.daemonology.net/blog/2015-08-13-null-is-a-bad-idea.html <comment><USER2> _anti-pattern known sometimes as "defensive programming": If something goes wrong, pretend you didn't notice and try to keep going anyway_

No. Defensive programming means, be predictable even in the face of nonsense. Return an error code instead of segfaulting, define reasonable non-error behavior for thing that don't _have_ to be errors, etc. Sweeping things under the rug just means they'll come back to bite you later, which isn't defensive at all.

 _and when faced by a choice between making the presence of a bug immediately obvious or trying to hide the bug and potentially exposing serious security vulnerabilities... well, I know which I would prefer_

This over a feature that makes "printf debugging" more usable? Is he smoking something?

If the security of your code relies on a parameter not being null, you'd be a fucking idiot to let "printf() should segfault" be the way you catch that. Check it explicitly; not only is it safer but it makes broken callers easier to debug.

<comment><USER3> It can be difficult to let go of printf(3) debugging, but using a debugger instead can seriously improve your development workflow.

<comment><USER4> Not for multi-threaded or networked code where breaking in the debugger changes the program's behaviour and causes timeouts. I still prefer printf in these cases.

<comment><USER5> As a Ruby and C developer, I wish there were more tools like Java's _jvisualvm_ for realtime visualization of running systems without the overhead and forced synchronization of a debugger.

The next best thing IMO is detecting crashes and printing a stack trace, which reminds me I've been meaning to release some code I wrote that does so for C/glibc programs.

<comment><USER4> I completely agree about jvisualvm, I'm still blown away by the quality of the Java ecosystem. I've done a lot of server-side work on the JVM and it really blows away every single piece of tooling for C/C++.

While I'm no fan of the Java language, you get all these benefits for other JVM-hosted languages such as Clojure and Scala and that is fantastic. You basically get the entire ecosystem for free.

</comment></comment></comment></comment></comment><comment><USER6> Having "%s" print "(null)" is perfectly fine for debugging and actually makes your life a lot easier since seeing "(null)" in your output is much quicker than popping into the debugger trying to figure out what line caused a segfault. (Yes, you can do it with a ternery but writing rigorous debug code is never on anyone's list).

Having "%s" print "(null)" is absolutely useless in the meat of your string manipulation code. It's pretty much guaranteed to be a bug and could easily mask a problem (as described in the article).

So I think the real problem here is that printf() is used for both debugging and for non-debugging string manipulation (sprintf() and friends). It's trying to serve two masters and it can never get both cases right. Maybe we should make "%s" not check for null and add a null-able "%S" for debugging.

</comment><comment><USER7> I often use printf for debugging. An unexpectedly NULL string is exactly the sort of thing I want it to help me debug!

<comment><AUTHOR> Crashing works just as well as printing "(null)" for that purpose. Possibly even better, since you'll have a core dump. ;-)

<comment><USER7> Sometimes it does, but often it does not:

        printf("name: %s %s", first, last);

If it crashes, there's no easy way to know which one was NULL!

<comment><AUTHOR> Tell your debugger to open the core dump and print the two pointers. ;-)

<comment><USER8> Just try that when debugging an app on an embedded system, mobile, or a console, where you have access to the console but no core dump.

Sorry, but I agree with all the folks who say printf (and similar) is categorically more useful when it prints "(null)". I've had _many_ obscure crashes on systems that were precisely because a logging printf in a seldom- tested piece of code had a parameter that sometimes was _correctly_ null would crash it out. If the debugger isn't attached and we just have a crash dump we can't read (happens even on platforms where you're SUPPOSED to be able to read them), then we have to attach the debugger and hope that we can reproduce the obscure state that caused the crash.

Anything that makes debugging statements more robust and less crash-prone is a _good thing._ End of story. As others have said, if you want something to verify that a pointer is non-null, use an assert().

<comment><AUTHOR> _As others have said, if you want something to verify that a pointer is non- null, use an assert()._

I said that, too. But sometimes developers don't think to put in every possible assert. Having the C library explode violently when it detects a bug in their code _even when they forgot to include an assert which checks for that bug_ will help to find bugs under such circumstances.

<comment><USER9> In the environment under question, you do not need an assert. You just have to dereference the pointer. Unless the pointer is displaced past the unmapped page zero (i.e you're accessing str[4096] and above, and not touching str[0] through str[4095] on a system with 4K pages) you will get a fault resulting in a fatal signal: SIGSEGV. Whereas an assertion will get you a SIGABRT, but the result is the same. The assertion will be earlier, that's all, before you try to dereference the pointer. Probably not much earlier.

Environments that do not have the hardware-assisted detection for null dereferences can still be targetted by a compiler which generates code which does the check on pointer dereference. That can be included in your debug builds, or even left in production builds if the performance impact is acceptable. I seem remember that some compilers for the Intel 8088 MS-DOS environment had a code generation option for trapping nulls.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> This is nonsense. There is no reason why printf has to be the sacrificial function that blows up on null. Those same "misguided" operating systems run on hardware that traps null dereferences via an unmapped page. A predictable, tidy crash is just around the corner if that same is used in the program. On the other hand, if that printf is the _only_ place the pointer is used, then there is no security hole. It's just undefined behavior on ISO paper, and you don't agree with how it is locally defined by the implementation.

How do we know it's a security hole? Maybe some structure has a "char * " name field, and null indicates "anonymous/has no name". It's an ISO C conformance bug to try to print that with %s, but that's a portability issue. If that's the only problem in the program (everywhere else it checks for null), there is no real issue.

POSIX and C libraries are not entirely about allowing null pointers or not. For instance gettimeofday lets you pass a null pointer for the "struct timezone * " second argument, which indicates that you aren't interested in specifying or retrieving it. On the other hand, a null argument is undefined behavior in getgroups, even if the array size is specified as zero (meaning, return me the size of the array actually needed and I will call you again soon with such an array)! A non-null pointer must be passed as a POSIX-conforming eggcorn. According to the logic of this article, if I pass null I'm committing a security hole, and an OS which allows getgroups(0, NULL) is misguidedly sweeping it under the rug.

Null pointers are not some "bogey man". I built a Lisp dialect in whose implementation a C null pointer represents the nil symbol (empty list, false, ...). It prints as "nil", satsifies the symbolp function and so fort. Lexical variables are initialized to nil if not given an initial expression. Some C library printing a null pointer as (null) seems like nothing by comparison.

Be that all as it may, you cannot actually _fix_ this perceived issue by changing from (null) to crash, cold turkey. Unknown numbers of nonportable programs out there (otherwise well-tested programs) which rely on this will just crash.

I just did a "grep -r -F '(null)' /var/log" on a system here and found a few hits. Most of those are probably via the %s conversion, and all that code would have to be fixed if printf were to assert this. And that's just instances confined to logging, not all output everywhere.

<comment><USER10> >Maybe some structure has a "char * " name field, and null indicates "anonymous/has no name". It's an ISO C conformance bug to try to print that with %s, but that's a portability issue. If that's the only problem in the program (everywhere else it checks for null), there is no real issue.

I'll take 'uses of Option/Maybe' for 200 Alex

Some might say it's overkill, but if you can have a char* be null, then for most intents and purposes you have two states to deal with on any usage of that value.

Just because such a design can exist doesn't mean we should hobble our tools to support it.

<comment><USER9> It could simplify things elsewhere.

In Lisp, CAR and CDR accessors take a cons cell or NIL, which isn't a cons cell, and has no such fields.

It wasn't always like that; according to the HOPL paper on evolution of Lisp, InterLisp introduced this nicety and it was copied by others.

These functions are complicated with that extra state, but programs are simplified.

A null "char *" indicating "this object has no name" is better than some special string value doing the same, because it's different from every possible string value.

<comment><USER10> I'm not saying a special string value, but a wrapper object with an "existence" bit. You can then assume, if the existence bit is flipped right, that the char* "exists".

This is functionally equivalent, but forces existence checks when they're needed. And by removing "null" from your vocabulary, an entire class of errors disappears.

With C this is a bit trickier to ensure. But I think the design pattern can still be useful. And it doesn't make code more complicated, but rather it makes the complicated aspects explicit . Instead of testing that name == null, you test that name.exists is False, and if not you use name.value. And you make null a banned word in your codebase.

The result is that you _never_ have to ask "is this char* possibly null?". The answer is always "no". So you remove superfluous checks

I think this pattern works much better when you have stronger type checking, though...

</comment></comment></comment></comment><comment><USER11> "This is an example of an anti-pattern known sometimes as "defensive programming": If something goes wrong, pretend you didn't notice and try to keep going anyway."

That's why SQL NULL is also a bad idea.

<comment><USER12> SQL NULLs propagate; you can't get an answer from a NULL unless you explicitly plan for it.

<comment><USER5> It can be a bit counterintuitive that "([NULL] < x)" and "NOT ([NULL] < x)" both return NULL in SQL.

        MariaDB [(none)]> select null as a union all
            select 1 as a;
        +</comment></comment></comment><comment>+
        | a    |
        +</comment><comment>+
        | NULL |
        |    1 |
        +</comment><comment>+
        2 rows in set (0.01 sec)

        MariaDB [(none)]> select * from (select null as a
            union all select 1 as a) T where a >= 1;
        +</comment><comment>+
        | a    |
        +</comment><comment>+
        |    1 |
        +</comment><comment>+
        1 row in set (0.00 sec)

        MariaDB [(none)]> select * from (select null as a
            union all select 1 as a) T where NOT a >= 1;
        Empty set (0.00 sec)

</comment><comment><USER13> "If you accidentally pass a null pointer as the argument for a '%s' conversion, the GNU C Library prints it as '(null)'. We think this is more useful than crashing. But it's not good practice to pass a null argument intentionally."

[https://www.gnu.org/software/libc/manual/html_node/Other- Out...](https://www.gnu.org/software/libc/manual/html_node/Other-Output- Conversions.html)

It's documented.

<comment><AUTHOR> There are more C libraries than just GNU C...

<comment><USER9> If there weren't, we wouldn't need a library clause in the ISO C standard to know what is portable among them. That doesn't add up to an argument that libraries shouldn't have documented extensions in areas left undefined by the standard.

</comment></comment></comment><comment><USER9> From BSD errno manpage:

         14	EFAULT Bad address.  The system	detected an invalid address in
    	       attempting to use an argument of a call.

Is that an example of a misguided operating system? The kernel should kill the offending process instead of sweeping the bug under the rug with an errno code that might not be examined.

Those who live in glass houses ...

<comment><USER14> I have not checked which syscall or libc function could return that, but it's quite possible that some applicable standard like C or POSIX or SUS mandates such a return value.

It is better to follow the standard in such cases, so that portable code written against the standard runs nicely.

However (claims the OP) in the case of printf, the %s null pointer case is explicitly defined to be undefined behavior and hence the implementation should handle this either implicitly (by crashing with null pointer dereference) or explicitly by calling abort(), as that will teach programmers not to rely on superficially benign implementations of undefined behavior.

<comment><USER9> > _mandates such a return value._

And ISO and IEEE standards can't be "misguided", just competing operating systems that are more successful.

> _explicitly defined to be undefined behavior_

Contradiction.

Undefined behavior is not defined, explicitly or otherwise. There can be explicit wording that some _requirement is not being made_.

When ISO C explicitly states that there is an absence of a requirement, that does not carry any more or less weight then when behavior is undefined by omission of any wording of a requirement. All UB is equal.

> _not to rely on superficially benign implementations of undefined behavior._

An excellent example of beneficially fixing undefined behavior is to implement strict left to right ordering on all operations and their side effects.

A language or language dialect in which evaluation order is defined is safer and better for engineering.

<comment><USER14> > And ISO and IEEE standards can't be "misguided"

That's not a claim that I would ever make. But the right time to bring up objections is during the drafting of the standard.

If you aim to implement a standard you should have a very good reason before you implement some interface differently than specified. I would argue that these return values are not a good enough reason. gets(3) on the other hand would hardly be missed...

> Undefined behavior is not defined, explicitly or otherwise. There can be > explicit wording that some requirement is not being made.

In C, undefined behavior has a quite specific meaning: invoking undefined behavior is an error in the program and the implementation is at liberty to do anything it pleases.

 _" undefined behavior: behavior upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements."_

Your example of evaluation order is not undefined behavior, but unspecified behavior: there is a set of possible orderings and the implementation may pick any one of them.

<comment><USER9> _In C, undefined behavior has a quite specific meaning: invoking undefined behavior is an error in the program and the implementation is at liberty to do anything it pleases._

That doesn't match my understanding, which is that undefined behavior is about erroneous constructs or data, or _non-portable_ constructs.

Oh look, you quoted it. Thanks!

Some undefined behaviors are important sources for useful, documented extensions.

</comment></comment></comment></comment></comment><comment><USER15> So is "is a bad idea" replacing "considered harmful"?

<comment><AUTHOR> Huh. Somehow the words "considered harmful" never even occurred to me while I was writing that.

</comment></comment><comment><USER16> "on some misguided operating systems" — on which well-thinking systems _does_ printf crash in this case?

</comment><comment><USER17> I think it's a Bad Idea in C/C++, but it may not be a Bad Idea in some untyped interpreted languages, or in a special "checked build / debug" C++ environment.

</comment><comment><USER18> In the context of where this would be a bad idea (e.g. flight control, rescue rover, etc), you would not use `printf()`, or perhaps any standard output at all. If anything, you will use lower level `write()`, where EFAULT or something would be raised on NULL, and/or static buffers, where NULL is impossible. `printf()` is really only good for developer-friendly programming, where absolute rigor is not the focus.

<comment><AUTHOR> _`printf()` is really only good for developer-friendly programming, where absolute rigor is not the focus._

printf gets used all the time in systems code. I count 1221 occurrences of printf (and friends) in BIND 9 alone.

More to the point, "(null)" was just an example of a debugging-hostile pro- vulnerability attempt to silently "fix" bad data.

<comment><USER4> BIND 9 doesn't have the same requirements as flight control software. Nobody will die if BIND segfaults, at least I hope not.

The software running in your car, plane, spacecraft and the likes have very strict regulations and you will find it radically different from systems code found in a linux distribution, even if written in the same language!

See [http://lars-lab.jpl.nasa.gov/JPL_Coding_Standard_C.pdf](http://lars- lab.jpl.nasa.gov/JPL_Coding_Standard_C.pdf) for more details. I wouldn't want to write code like this as it sounds very unproductive, but their reliability and astronomically small number of bugs speaks for itself.

<comment><USER19> BIND is important internet infrastructure. We'd all be better off if it did follow the JPL standard.

<comment><USER4> Important yes but BIND can also afford having crashed nodes without the internet coming down to its knees. I'd be surprised if such a node came down and caused damages in the hundred of millions of dollars in the process. This is the distinction I wanted to make.

I can't remember the last time my DNS didn't work. The internet can afford to be 'good enough', life critical software can't.</comment></comment></comment></comment></comment>


Indexing the Andreessen Horowitz Portfolio Companies - <AUTHOR> http://www.daniellemorrill.com/2013/04/indexing-the-andreessen-horowitz-portfolio-companies/

<comment><USER2> If you are interested in aggregating their google page rank you can look at my code here: [https://github.com/databigbang/enriching-a-list-of-urls- with...](https://github.com/databigbang/enriching-a-list-of-urls-with-google- page-rank) using <http://www.schurpf.com/google-pagerank-python/>

Also, if you want to aggregate its related twitter, blogs, etc you can also look at <https://github.com/srw/discovering-web-resources> for a simple multithreaded crawler.

Related articles:

\- [http://blog.databigbang.com/automated-discovery-of-blog- feed...](http://blog.databigbang.com/automated-discovery-of-blog-feeds-and- twitter-facebook-linkedin-accounts-connected-to-business-website/)

\- [http://blog.databigbang.com/enriching-a-list-of-urls-with- go...](http://blog.databigbang.com/enriching-a-list-of-urls-with-google-page- rank/)

Hope it helps!

<comment><AUTHOR> Thanks, I will check these out and really appreciate the interest in what I'm doing

</comment></comment><comment><USER3> _> Andreessen Horowitz has an impressive list of exited companies in 3 IPOs (Facebook, Groupon, Zynga) and 3 acquisition from high profile companies: Nicira (acquired by VMWare), Instagram (acquired by Facebook), and Skype (acquired by Microsoft). _

Hi Danielle, FYI Fusion-io (under enterprise) has exited via IPO: <https://www.google.com/finance?q=NYSE:FIO>

<comment><AUTHOR> Awesome, I will remove them from the list

</comment></comment><comment><USER4> This is missing their seed stage investments

<comment><USER5> That would be a very useful addition.

</comment></comment><comment><USER6> I love this index series, Danielle! Does this have any link to what's next for refer.ly? :)

</comment><comment><USER7> This ranking under-states mobile companies quite a bit. I'm not sure the best way to improve it, but I guarantee you that Bump and any TinyCo game has broader reach than App.net or Viki.

<comment><AUTHOR> I absolutely agree, I am working on this

<comment><USER8> Use the Play store/App store rankings and download counts as a factor and weigh them higher for mobile-only companies?

</comment></comment></comment><comment><USER9> Hey dmor, mind updating "Balanced Payments" to just "Balanced" website is balancedpayments.com, but company name is Balanced. thanks!

<comment><AUTHOR> Fixed, thank you

<comment><USER9> Thank you

</comment></comment></comment><comment><USER5> Very useful. Can you do other VC's as well?

</comment><comment><USER10> Great post! Just noticed a typo though - it's 'foursquare' (single word, all lowercase), not 'Four Square'.

<comment><AUTHOR> Fixed, thanks</comment></comment>


Confluence is broken. Here's how I plan to fix it - <AUTHOR> https://medium.com/@francescolanciana/confluence-is-broken-heres-how-i-plan-to-fix-it-ffb736fd2b28 <comment><USER2> Normally I would not promote my own company to the HN community without contributing something of value, but from the looks of the UI decisions in the "juicy screenshots" I would venture to say we have already done so: [https://share.slab.com/v1uy96K8](https://share.slab.com/v1uy96K8)

So if Confluence and internal knowledge sharing is problem for your team but you want to try out a ready and available product, please give [https://slab.com](https://slab.com) a look.

<comment><AUTHOR> Cheeky, but all is fair in business I guess. I had a look at every product I could possibly find (Notion, Confluence, Slab, Bear, CodeStoryApp, Evernote, Dropbox Paper, Google Docs, GitHub Wiki, Quip, Guru, Slite, Coda... listing them out makes me anxious) and made sure to incorporate what I thought were really smart product decisions. From Slab I really liked the table of contents and having the search bar up the top left, I can't say much more was included in Scribe that doesn't exist in every other product tackling this space though.

</comment></comment><comment><USER3> I'd love for any of these tools to offer :

#1 Explicit Document ownership. The primary author, as well as team distro list slapped at the top.

#2 Document aging. If a doc goes over X days without review by the author or team, it is listed at the top in yellow/red/etc as some level of "Stale" so others know. This also has the slight shame effect if you're the primary owner and all of your docs are bright red and stale.

#3. Directory structure template that you enforce on your entire org. So when a new team starts, you can have some sort of standardization stamped out around where things are like Design Docs, Runbooks, Meeting notes, etc, etc

<comment><USER4> I really like the idea of #2. Maybe combined with some way to indicate the 'lifespan' of a piece of documentation at the time of creation. We've got bits of docs lying around that were intended to be updated weekly, and regularly fall to the wayside for months at a time. A way to just tag them as 'weekly' and have them pop to the top of a list in yellow/red when they haven't been updated in 7/10 days, would be useful.

Even if it just ends up being a trigger for people to go 'hey, I cbf updating this anymore' and deleting it. It's better than just keeping it around in limbo forever. Our confluence is a graveyard of useless pages and it's only going to get worse over time. I imagine most are the same.

<comment><AUTHOR> I'm glad to hear I wasn't the only one experiencing this. When using the Scribe feature I mentioned to mdeeks if anything becomes outdated that was under your purview you will see it at the top of your feed. Didn't want to go down the notification route because I was worried you would get 100 notifications and just learn to ignore them

</comment></comment></comment><comment><USER5> Confluence is objectively horrible, without a doubt, but adding (yet) another candidate to the marketplace isn't really fixing Confluence. For starters, I'm still going to be stuck using Confluence.

I'm still sad at the death of DekiWiki / Mindtouch -- it was one of the best wikis I ever used, but was abandoned around 2010. It offered a structured hierarchy, a decent editor, was performant, looked good, self-hosted, scriptable extensibility, etc.

I've never found a satisfactory replacement.

<comment><AUTHOR> Maybe "Here's how I plan to rectify it" would have worked better... putting that aside you Jedd are in luck! Scribe isn't just any candidate, it's the candidate (If I can get enough people to throw money at me so I don't have to go get another job). I'll have a look at DekiWiki and Mindtouch tonight. What did they offer in terms of structured hierarchy that other tools don't? Also interested to see what this scriptable extensibility is.

I do agree that getting Enterprises to buy into my tool over Confluence will be really difficult, and until then I'll struggle to help you, but I'm lining up as many talks with people that can make that call as possible to see what I would have to do to make that a reality. One thing you might find interesting is that I plan to make Scribe free for public wikis, so while it might take some time for your team to use it, it might be useful to you if you do open source work? And if you like it enough maybe you can recommend it to your team, who knows!

<comment><USER5> Hi Francesco - happy to reminisce.

Refer the wayback machine, as Mindtouch took down their documentation a while ago. Short of running up a live copy [1] on a VMware instance, the fastest way to get an idea of the features is to review their user & API documentation [2].

The sourceforge site has some screenshots, but I suspect there are very few publicly accessible instances anymore, as the thing was so damned hard to patch (VM images were how they distributed the software). I did find this [3] but you won't get to experience editor / API, only the navigation.

Deki's hierarchy worked - most other products I looked at this was an effort (plugins, extensions, etc) - either way it prevented orphan pages, and reduced the risk of losing pages or people accidentally creating duplicates.

The hierarchy was also reflected in the URL, which is just lovely (compare Sharepoint). Confluence has a hierarchy, but the URL does weird things, and I think there's a global namespace for pages IIRC. Their URL's _sometimes_ go a bit whacky, but haven't worked out why.

[1] [https://sourceforge.net/projects/dekiwiki/](https://sourceforge.net/projects/dekiwiki/)

[2] [https://web.archive.org/web/20120828154742/http://developer....](https://web.archive.org/web/20120828154742/http://developer.mindtouch.com/en/docs)

[3] [http://www.osslab.tw/](http://www.osslab.tw/)

</comment></comment></comment><comment><USER6> Cool app. I agree with your points about Confluence's issues.

> At this point documentation becomes viewed as a time sink and people start > to prefer other practices like pairing, comprehensive user stories and > tests. These unfortunately are only a bandaid solution to the problem...

I think that "pairing, comprehensive user stories, and tests" are all good things you should have anyway, and documentation can't be a replacement for them.

<comment><USER7> Can someone explain what "pairing, comprehensive user stories, and tests" are? Are these actual terms or is the author just talking about teammates getting together to work on an issue?

<comment><USER8> I assume they mean pair programming [1], user stories which are used in agile methodologies [2], and test-driven development [3].

[1] [https://en.wikipedia.org/wiki/Pair_programming](https://en.wikipedia.org/wiki/Pair_programming)

[2] [https://en.wikipedia.org/wiki/User_story](https://en.wikipedia.org/wiki/User_story)

[3] [https://en.wikipedia.org/wiki/Test- driven_development](https://en.wikipedia.org/wiki/Test-driven_development)

</comment></comment></comment><comment><USER9> I worked with confluence, google docs and git like repos. Wholeheartedly prefer the last one.

<comment><USER10> Git like repo? Could you please elaborate?

<comment><USER11> writing a lot of markdown docs. ;)

edit of a new page/section is a PR to the repo. once merged, you can auto deploy it to the target destination, rendering via jekyll or mkdocs or other things.

Discussions happen in PRs in that model. (you can also added previews of the PRs and deploy these next to the production version of the site)

<comment><USER11> the better your git repo tool supports online editting the better this experience will be.

</comment></comment></comment></comment><comment><USER12> It is wiki nature to be an ephemeral snapshot of documentation in time. To ask it to be otherwise is like asking "a minute ago" to always be up to date.

Discoverability is hard. Build a better mechanism for that and you could set your sights higher than a better wiki.

I can't understand how the rest of the enumerated points is a problem with wikis per se. Or what the numbers assigned to those points mean, for that matter? (Ranking? Reference? Decoration?)

<comment><AUTHOR> I agree that a wiki will never be truly up to date, however I do think it's possible to build in mechanisms that tell people what is and isn't up to date. For example if a code snippet you include in your document changes (i.e. it's updated in GitHub) then the document should be either marked as outdated or in need of review.

As for discoverability you are right, it's definitely hard. My idea here is to allow people to create teams on the fly. You then have a feed that your team members and manager can push content to and manage.

Oh the numbers didn't mean anything, I was just trying to make it easier to read. Will see if I can reword it to make it clearer :)

As for them not being a problem with wikis, I think they are if you look at the wiki as not just being a collection of documents but as being a living breathing knowledge base. Interaction and collaboration are important to encourage if you want people to feel like their documents are actually being used. Even a simple clap button like Medium has would go a long way in this department. Insights into how your document is used would help show you how to refine the document, and also feeds into your work being valued. Plus it might really help managers gain a better understanding of the knowledge gaps that exist. Documentation becoming outdated is a reality that I feel most tools out there ignore, but addressing it is really important in having people trust what they read (I know my trust in my teams documents dropped off pretty rapidly after I kept finding them to be outdated). The last point was more of a thought I had two hours ago haha, and again really ties into having a living breathing knowledge base people can trust and want to use.

</comment></comment><comment><USER13> Confluence is broken and almost all the solutions that have been provided here is broken since they are not open source. They don't let you contribute and you have no open way of suggesting and/or coming up with another feature. Wikis are not WYSISWYG, some of the platforms our there are SaaS and you can't trust them fully to store your company wide data and this list goes on. This is why (shameless plug) we develop Balsa in an open manner to fix how we document things. Check [https://github.com/balsa- team/balsa](https://github.com/balsa-team/balsa) and start contributing.

</comment><comment><USER14> I'm excitedly waiting for Clubhouse Write as I couldn't get into the private beta but if it does as good a job at replacing Confluence as Clubhouse did at replacing JIRA for me then I'll be shouting it from the rooftops.

Much like JIRA, Confluence is incredibly powerful but effectively requires a full-time person taking care of it and organizing/cleaning things up. Unfortunately the scale at which Confluence breaks due to the number of documents and lack of organization is far lower to the scale that would warrant such a 'caretaker' role hence I've never seen Confluence used in a fashion that didn't get in everyone's way and exhibit all issues the article listed.

<comment><AUTHOR> Oh good another competitor haha. I really like the way you worded the scaling issue, couldn't agree more. When I release it I'm hoping you're still in a shouting mood.

</comment></comment><comment><USER15> Organizing at the document level is the main mistake that virtually all of these tools make. Search doesn't get around that mistake either.

<comment><AUTHOR> I'm really interested, what do you mean by that? How do you think they should be organised?

<comment><USER16> Not the original commenter, but he got me thinking. (I'd like to know what he soecifically meant too.)

However, I find that nuggets are often embedded in subsections of larger documents.

Perhaps the search could rank higher a document with subsection with query words in the title. (Maybe they do in some tools?) Any set of query words could be by chance present, but scattered in a hundred large documents not relevant for the user. Conversely the tiny specific subsection I'm looking for is lost in the noise.

This very thing happened when I was looking for the delivery and address information for a main office. Huge number of documents mentioning address of something and delivery of something showed up. I even remebered seeing an earlier revision of the subsection I was looking for, but it didn't help. IIRC even adding parts of the street address didn't help because the plain address was common in many documents for random reasons.

Personally I also like to divide my technical documents into subsections with exact titles. It would be great if tools knew how to exploit this kind of hierarchy.

</comment></comment></comment><comment><USER17> I don't think I understand. You say you're going to fix Confluence. That's great! Lots of us are stuck using Confluence, after all, and it has many deficiencies.

However, I don't really understand how your product integrates with or improves Confluence?

<comment><AUTHOR> I am quickly learning I need to spend more time wording things appropriately haha. To be clear it doesn't integrate with Confluence in any way, shape or form. Scribe is really a direct competitor to Confluence. But it will fix all the deficiencies that Confluence has.

</comment></comment><comment><USER18> I don't think I understand how Confluence (an awesome tool) is flawed and how adding a page view counter to a wiki is going to fix all the "problems" listed

<comment><AUTHOR> Hey could you give it another read? I've added a bunch more detail about what the product actually does to try and tackle these problems. Thanks for the feedback, I somehow completely overlooked the fact I didn't tell you what it did haha

</comment></comment><comment><AUTHOR> To everyone that gave me your email, I appreciate it massively. Look forward to some great (and I promise non-spammy) updates :)

</comment><comment><USER19> Dear Franceso and others, please stop writing "X is broken" when it is clearly not.

I don't see any point in reading an article stating something that is untrue, in this case Confluence which if it was broken would not have hundreds of thousands daily users nor a developer raking in cash from new licenses.

If you want my interest or click, say "How to make Confluence 2x better", "A better alternative to Confluence" and so on.

Because if you want me to invest in you or your product, don't start by lying to me. It'll only makes me question what else you are not honest about.

<comment><USER20> Oh come on, don't get caught up in the word games, it's unproductive. You know the author meant something more specific than "X is broken" because he wrote an entire article explaining what he means when he says "broken".

This is just engaging in the conversation in bad faith...

<comment><USER19> I understand what you are saying but my intentions are good and I'm trying to help.

It's a way smaller chance that I'll click a link or read an article if it starts off with something that is wrong, instead being clear and precise in your communication is IMO a huge advantage when trying to attract me as a customer.</comment></comment></comment>


Climate Bill, R.I.P. - <AUTHOR> http://www.rollingstone.com/politics/news/17390/183346?RS_show_page=0

<comment><USER2> I work in the mining industry and this is huge. Great news. Global warming is a complete scam anyway. The planet naturally goes through cycles of hot/cold. To think in the microscopic amount of time that humanity has been industrialized, that we could actually affect the climate of our planet, is ludicrous.

</comment><comment><USER3> Global warming is big business(draws in millions) and it's a hip and popular movement. Is it for real though?

I'm on the fence, as we had 84 inches of snowfall here in the east coast this winter. Record breaking snow falls for these parts. Yet this summers' temperatures have been insane; always 90 degrees on up to 100.

<comment> bjelkeman-again "Is it for real though?", I take as you don't think you have seen enough evidence in the data to support the thesis that climate change is happening or that it is caused by humans.

I suggest you start here: [http://www.skepticalscience.com/empirical-evidence- for-globa...](http://www.skepticalscience.com/empirical-evidence-for-global- warming.htm)</comment></comment>


How Terms & Conditions legalese should be written - <AUTHOR> http://octopart.com/legal

<comment><USER2> "(iii) deep-link to any portion of Our Website (including, without limitation, the purchase path for any search services) for any purpose;"

No deep-linking? wow. (Not quite sure how they enforce that rule)

<comment><USER3> One could enforce it by looking at the "referer" field in the http headers (yes, we know that it _should_ be spelled referrer, but it is mispelled that way in the standard/rfc). If the referrer is from "somewhere else" then they could redirect the user to the front page.

This is similar to how many websites, such as Wall Street Journal, will let non-subcribers read an article if the user came in via a Google search (essentially, if the http-referer field = google, then the subscriber check is bypassed).</comment></comment>


Ask HN: Good examples of social networks with a game mechanism? - <AUTHOR>

Rewards, achievements and game mechanisms in social networks is a strong force to get people engaged. Foursquare and Gowalla are two great examples of this.<p>So, I'm currently looking for more examples of this and how you can make this type of mechanisms to work. Any thoughts or examples out there? <comment><USER2> Of course, you're on a social network right now which uses game mechanisms.

My understanding is the entire x-box live is based on game mechanisms.</comment>


Responsive Dashboard - <AUTHOR> http://ehesp.github.io/Responsive-Dashboard/

<comment><USER2> Compare this to any admin template from WrapBootstrap and you know why this is at the top.

If you look at a WrapBootstrap template, it generally looks nice -- but as soon as you look at the code, your opinion will change. They literally include any popular JavaScript library that exists, just to show off what's possible. And the code is not well-written, either. 70 HTTP requests and 5MB resources to load is the new normal there.

This one here, in contrast, is clean, polished, and shines due to its simplicity.

<comment><USER3> Thanks! That was the idea.

</comment></comment><comment><USER4> I like it, it looks really great!

However, the html offers a couple of great chances to use angular directives or some databinding. Especially the custom directives offer a great way to show the semantics or the intent of what the page is all about, so that in the end the dashboard could read somthing like:

       <overview>
         <users></users>
         <servers></servers>
         <documents></documents>
         <tickets></tickets>
       </overview>

       <server-list></server-list>
       <user-list></user-list>
       <extras></extras>
       <...></...>

Something like that; you get the idea.

Keep up the great work!

<comment><USER3> Hey, I'm the author of this and never knew this was posted on here.

I'm really a backend developer so my knowledge of Angular really isn't that great. I'll look into what you suggested, however I didn't want it to be too complex; quick and simple way to get up and running was the idea.

<comment><USER5> Hey aliasell, I'm actually developing a similar dashboard at my company. I'm a full stack developer, feel free to PM me github.com/Igosuki

</comment></comment></comment><comment><USER6> This looks nice, but I do have some quibbles of a different nature.

[https://github.com/Ehesp/Responsive- Dashboard/blob/master/js...](https://github.com/Ehesp/Responsive- Dashboard/blob/master/js/angular/bootstrap.js#L59) \- this is an expensive operation. It would be better to use window.matchMedia and do a $scope.$apply() only precisely when needed. [https://github.com/Ehesp/Responsive- Dashboard/blob/master/js...](https://github.com/Ehesp/Responsive- Dashboard/blob/master/js/angular/bootstrap.js#L25) is also expensive too, since getWidth is a function.

My other criticism is that it should avoid adding ngCookies as a requirement, as it is a not so great portion of angular.

I think some of the people criticizing this miss the point of something like this. It is a theme that is geared specifically towards people who use Angular & don't want to use jQuery. If you don't use Angular/want to use Angular, this isn't for you - or you can fork it and port it over.

<comment><USER3> Hey, thanks for the suggestions appreciated. I'll dig into the width and apply areas, my JS isn't my strongest area so the implementation came from a "what works" situation.

</comment></comment><comment><USER3> Thought I'd leave this here. I'm Elliot (owner of the repo).

The dashboard isn't intended to be driven by Angular, it's more of a basis of getting going on a project without messing about with the initial setup - however have a 'clean' boilerplate to work with without the masses of plugins all of these premium dashboards come with.

I'm also not a pro coder, and do it for a hobby so comments on improving are much appreciated. It's free, open source, I'm massively open to people improving my code so if you can please do. I'm currently learning Angular so appreciate it may not be coded to a specific standard I don't know about.

</comment><comment><USER7> I for one have starred this. Instead of being assholes, create some PRs. The notification right at the top says:

"Feel free to create pull requests to improve the dashboard!"

</comment><comment><USER8> Here's the project's home on Github: [https://github.com/Ehesp/Responsive- Dashboard](https://github.com/Ehesp/Responsive-Dashboard)

</comment><comment><USER9> If you want a quick overview of what this looks like on various devices: [http://ami.responsivedesign.is/?url=http://ehesp.github.io/R...](http://ami.responsivedesign.is/?url=http://ehesp.github.io/Responsive- Dashboard/)

<comment><USER10> Funnily enough that site looks terrible on an iPhone...!

</comment></comment><comment><USER11> Awesome dashboard and project! Bookmarked to come back to it and use it :-)

P.S.: Wish the title was not so poorly chosen. Remember that title really does matter when submitting anything to Hacker News.

</comment><comment><USER12> While this looks good aesthetically, this is a prime example of needlessly using JavaScript for the hell of it. This is like an anti best practice. It loads 4 extra resources (Angular, Angular Cookie, Angular UI Bootstrap and the custom bootstrap.js) for no legitimate reason. And the CSS is inexplicably written using selectors like the following: "#page-wrapper:not(.active)". Why not target "#page-wrapper" and then "#page-wrapper.active" instead?

<comment><USER13> Your CSS quibble is a matter of taste, you have a legitimate complaint about the JS so I wouldn't muddy it with a taste based comment about CSS verbosity.

</comment></comment><comment><USER14> Where's the Angular part? None of the links seem to work..

<comment><USER15> Or the responsive part for that matter. Nothing is working for me on Firefox 31.

<comment><USER8> It's responsive for me in FF 31 (OS X) and in Chrome. When the window size is reduced, the content boxes reflow from multi-column to single-column, and the navigation on the left goes from text+icon to icon only.

</comment></comment></comment><comment><USER16> Not to criticize this project specifically but I was task some weeks ago to implement a design with a unfoldable menu like that.

Clueless manager loved it (yeah animation !) but it's not great. In practice you never want to unfold it because it doesn't give you anything other than labels. So then you are asked to display labels on mouse hover when it's folded.

And then you must add submenu ...

<comment><USER3> Not at all. I know what you mean, however a tooltip is easy to add to the labels - issue is this won't solve the issue for mobile devices.

As for sub-menus, I never needed it for my project so never added it... and honestly didn't expect anyone else to see this. Will have a tinker with it though, see if I can make it look decent.

</comment></comment><comment><USER17> Not really much AngularJS code?

<comment><USER18> Yeah. There's really no reason to put Angular in the title at all.

</comment></comment><comment><USER19> Hey this is a dumb question, but what exactly does "responsive" mean here? Is this a technical HTML/Javascript term? Or does it mean simply "fast"?

<comment><USER20> Not a dumb question. Responsive design is a modern web development technique for building sites that adapt to the screen on which they are being viewed. The idea is to provide a good user experience on mobile, desktop, and tablet with only a single web site (as opposed to detecting mobile/tablet devices and redirecting to a mobile-only version).

To see it in action, go to the link in the OP and just resize your browser window down to tablet size, and then down to mobile.

This is accomplished primarily through CSS3 media queries and fluid layouts (using percentages for element widths instead of fixed widths). You can read about CSS3 media queries here:

[https://developer.mozilla.org/en- US/docs/Web/Guide/CSS/Media...](https://developer.mozilla.org/en- US/docs/Web/Guide/CSS/Media_queries)

Perhaps the best resource for implementing responsive design is the Bootstrap framework:

[http://getbootstrap.com/](http://getbootstrap.com/)

</comment></comment><comment><USER5> This looks EXACTLY like the Dashboard I've been building at my company fot the past 6 month, wtf ?

<comment><USER3> No idea :p

</comment></comment><comment><USER21> God, these comments are why people hate Hacker News.

If you want to post an awesome project like this one, better do it somewhere else, if you want to be encouraged to keep working on it.

The design is absolutely gorgeous, for what it's worth.

<comment><USER22> I agree it looks fine (though it's not exactly April Zero...), but I think the comments below legitimately address some of the issues with the JavaScript (60 or so poorly formatted lines, using Angular for no reason at all) and the CSS - and probably people are confused as to why this frankly somewhat amateurish project is currently at the top of HN.

From the title, I think you'd expect something that's cleverly using Angular to pull in data dynamically from disparate sources and plotting pretty charts from it, rather than just some static HTML and CSS.

<comment><USER23> >people are confused as to why this frankly somewhat amateurish project is currently at the top of HN.

Probably the GP's irritation comes from the same place as mine: needlessly abrasive remarks instead of encouragement and constructive criticism that a community like this should provide.

And why do people care so much that this is at the top of HN, as opposed to the bottom?

>I think you'd expect something that's cleverly using Angular to pull in data dynamically from disparate sources and plotting pretty charts from it,

That part is relatively easy to add, given the nice template this provides. Really, for me, creating this kind of layout is the time-consuming part. Pulling in data from some REST api is a relatively quick task.

<comment><USER22> > And why do people care so much that this is at the top of HN, as opposed to > the bottom?

Well, there are only so many hours in the day, and it's nice to see exciting and innovative projects (and articles) gravitate towards the top - that is, surely, how HN is supposed to work.

<comment><USER24> Well it is at the top. It's obvious that this is representative of the interests of HN, otherwise it wouldn't be there. Sorry it doesn't meet your standards.

You mention it being amateurish. A constructive response would be to take it, pull in some REST data and show everyone whats possible. Or you could just complain.

</comment></comment></comment></comment></comment><comment><USER25> its crap...whatz so angular in this? 45 points for this..really..!!!

<comment><USER26> There really isn't anything at all special about this, the jQuery equivalent wouldn't have made it to 5.</comment></comment>


Ask HN: What are some ridiculous dev stories? I'm collecting for a book - <AUTHOR> I am working on writing a free book that is collection of ridiculous programing stories.
I thought HN would be a great place to fish for stories.
If you are interested in sharing a story you can respond to this thread with the story or send me an email:  firstname.lastname@example.com
Obviously all stories will be professionally edited and wont be published until receiving permission from original writer.
Looking forward! <comment><USER2> I worked at a company that has a flagship product that is a web-based spreadsheet.

The company uses huge spreadsheets to manage their network infrastructure.

These spreadsheets are so large that their own flagship web-based spreadsheet cannot handle them, so there's a whole section of the company running Excel on Windows workstations to manage them.

I worked on a replacement project for that.

It was also a web-based spreadsheet.

It was about three years old when I joined the project and I worked on it for two more years (before quitting.)

Punchline: It also could not handle the large spreadsheets.

(Sad truth: tens of millions of dollars and tens of thousands of dev-hours wasted with zero payoff. PMs getting promoted up and out and leaving the project behind resulted in a classic "Lava Flow"[1] mess. You could actually see the layers in the code and VCS.)

[1] [https://en.wikipedia.org/wiki/Lava_flow_(programming)](https://en.wikipedia.org/wiki/Lava_flow_\(programming\))</comment>