Time-Lapse Mining from Internet Photos - <AUTHOR> http://grail.cs.washington.edu/projects/timelapse/

<comment><USER2> This is an interesting counter-argument to the question "why bother taking photos of the well-known place X, there are already thousands of (better) pictures of it?"

</comment><comment><USER3> That's really interesting that the Wall Street bull moves over the years. I wonder if this is due to people leaning against it, or something like cleaning, or some other reason.

<comment><USER4> That one is in the "failure cases" segment

<comment><USER5> The blurring is the failure case, not the movement of the statue

</comment></comment></comment><comment><USER6> Very interesting project. They show a few "failure cases" at the end of the video; I'd be interested to see a few more of those. The ones they showed really didn't look all that bad.

I wonder what the copyright on such a timelapse would look like?

<comment><USER7> Some of the photos they used are CC licensed, so it would probably be CC-BY- SA.

<comment><USER6> My problem would the images that are copyrighted, unless they specifically pulled only CC images, which would limit photo numbers by quite a bit.

</comment></comment></comment><comment><USER8> This is great work that seems immediately useful to areas like conservation and art.

The images were heavily homoginised which was good for continuity but perhaps obscures details.

Using the timelapses as a scrubbable index to the original images would be a cool tool.

<comment><USER2> The images probably needed this homogenization because they don't match 100%, being taken from slightly different places and angles (so you not only get different framing but also parallax issues).

</comment></comment><comment><USER9> This reminds me of [https://vimeo.com/63653873](https://vimeo.com/63653873), where they used Google Maps Street View to make hyperlapses.

</comment><comment><USER10> The video displays a world map with the locations where they found enough photos to make time lapses. Does anyone has a link to this map? It would be great to discover the best time of the year to visit some places, like Salto Angel, the waterfall in Venezuela, or Lençóis, in Maranhão State - Brazil.

</comment><comment><USER11> Imagine combining this with Photosynth or whatever the project was that could stitch random photos into a 3D scene.

<comment><USER12> A lot of the underlying machinery is similar: they both rely on "structure from motion" (sfm) techniques to automatically estimate both camera locations and 3d geometry of the scene simultaneously. And both works come from the same lab: the GRAIL group at the University of Washington.

(I postdoced in that lab for 3 years and the authors of this paper are friends/former colleagues.)

<comment><USER11> Cool. SFM is a really fascinating concept to me. How much of that work is commercially available to consumers?

<comment><USER12> More than people realize... Google and apple, for example, use sfm heavily to compute their 3d maps (in Google earth and the apple equivalent). Google also uses it in various other products.

</comment></comment></comment></comment><comment><USER13> Very very cool, but I was expecting to see a time lapse of a mining operation and it's effects. Still want to see that!

<comment><USER14> have you seen manufactured landscapes?

[http://en.wikipedia.org/wiki/Manufactured_Landscapes](http://en.wikipedia.org/wiki/Manufactured_Landscapes)

[https://www.youtube.com/watch?v=o8cpCL8FSFM](https://www.youtube.com/watch?v=o8cpCL8FSFM)

beautiful film, there is one scene where the camera is tight on a 2 meter diameter tire dump truck and then slowly pans out until the dump truck is only the size of a pixel revealing a mining operation that fills the whole scene

<comment><USER13> Thanks for the suggestion!

</comment></comment></comment><comment><USER15> With all such results there is one question: are they releasing code? Otherwise... it's a nice demo to watch, but impossible to use (and even: hard to increment on it).

<comment><USER16> Well that'd be research code anyway :) You'd probably better off rebuilding from scratch based on the research paper. The software shouldn't be that complicated in the end, the main challenge (to me) is gathering that 86M images dataset.

<comment><USER15> Research code is infinitely better than no code. Research papers are almost never complete enough to reimplement it (seriously - I met a lot of academicians doing it, and most of the time it was not possible without contacting authors; just - text does not compile).

Vide:

[http://academia.stackexchange.com/questions/23237/why-are- pa...](http://academia.stackexchange.com/questions/23237/why-are-papers- without-code-but-with-results-accepted)

[http://academia.stackexchange.com/questions/37370/should- i-s...](http://academia.stackexchange.com/questions/37370/should-i-share-my- horrible-software)

(A similar things holds for data as well.)

</comment></comment></comment><comment><USER17> In retrospect this is a very obvious idea. Brilliantly done!

</comment><comment><USER18> Very impressive technology. This is just the beginning of leveraging the massive amounts of data out there in innovative new ways.

</comment><comment><USER19> I thought (inspired by Person of Interest TV show and photo supersaturation) after the Boston bombing it would be a good project to use publicly available photos, video, security camera video to track or search for suspects.

Or it could be used to search for a kid such as during an Amber Alert.

But I have no skills to do this :(

</comment><comment><USER20> There's a lot of news and 'new' stuff on the internet. And then occasionally you see something like this which is truly a fresh idea. I feel inspired when that happens!

</comment><comment><USER21> This is amazing! Would love to see more, and longer, timelapses!

</comment><comment><USER22> This shows how the natural world around us is slowly, but always changing. It never occurred to me that waterfalls changed like this!</comment>


A beginners guide to thinking in SQL - <AUTHOR> http://www.sohamkamani.com/blog/2016/07/07/a-beginners-guide-to-sql/ <comment><USER2> I think this would really benefit from some thoughtful indenting to illustrate the structure. For example, instead of:

      SELECT
      members.firstname AS "First Name",
      members.lastname AS "Last Name"
      FROM borrowings
      JOIN books ON borrowings.bookid=books.bookid
      JOIN members ON members.memberid=borrowings.memberid
      WHERE books.author='Dan Brown';

Do:

      SELECT
        members.firstname AS "First Name",
        members.lastname AS "Last Name"
      FROM borrowings
        JOIN books ON borrowings.bookid = books.bookid
        JOIN members ON members.memberid = borrowings.memberid
      WHERE books.author = 'Dan Brown';

This helps reinforce that there is one FROM statement that combines three tables using joins

<comment><USER3> Some time ago, I settled on a fairly similar idiosyncratic indentation style for SQL:

      select
        members.firstname as "First Name",
        members.lastname as "Last Name"
      from
        borrowings
        join
        books on borrowings.bookid = books.bookid
        join
        members on members.memberid = borrowings.memberid
      where
        books.author = 'Dan Brown';

Normally, I'm a fairly slavish follower of style standards, but almost every bit of SQL I come across in my job is an unreadable mess, even when whoever wrote it did have the vague idea that consistent formatting was good.

<comment><USER4> Sometimes with complex queries, it can be tough to adhere to a simple set of style rules while also making clear where things like logical units start/stop. Don't get me wrong I'm not disagreeing with you in principle; but acknowledging that the declarative nature of the language cause you to break style for the sake of readability in some cases. I'm not going to post an example of such a case (I'd have to go digging and I'm too tired).

On a different note, I don't like a lot of the styles I do see. I like yours, with the exception of the joins. My convention is a bit different:

      SELECT
          members.firstname as "First Name"
         ,members.lastname as "Last Name"
      FROM borrowings
        JOIN books
            ON borrowings.bookid = books.bookid
        JOIN members
            ON members.memberid = borrowings.memberid
      WHERE books.author = 'Dan Brown';

Some of that is old habit (like the comma first thing which I know is out of favor with many) which I found useful when working with long column lists. I'd also typically and systematically alias the tables, but that wasn't done in the examples. Outside of tastes, the only real downside I run into is that it's easy to get too far to the right with the indentation style I use. But it's not that often that I get there.

<comment><USER3> I haven't had trouble with complicated queries with this style. I've been doing it for ages, though.

There nothing wrong with comma-first, it's sensible; I just never got into the habit. My joins are so I can instantly glance over the tables involved in a query.

(And, oh yes, I, too, normally alias the crap out of tables if there's more than one in a query.)

</comment></comment></comment></comment><comment><USER5> I have to wonder why a relation algebra language isn't actually algebraic. What about projections in the form of something like c := a x b etc? Nooo, they had to make it look like COBOL.

<comment><USER6> In databases class, they used Pi for projection, bowtie for natural join, and sigma for row filtration. They added conditions on the operators with subscripts. SQL is just the standardized version of that.

<comment><USER5> "Standardized"...and without the legibility of symbolic notation and the option to use intermediate named relvars?

<comment><USER7> It's designed to be "natural", meaning that technical symbols were largely avoided. Someone who knows little about formal notation but a decent amount about databases can usually figure out what's going on. It's also easier to type, seeing how most of the relevant symbols aren't on your average keyboard.

</comment></comment></comment></comment><comment><USER8> What's the canonical reference book one should go to regarding SQL? (ie K&R is the book you go to for C)

I understand there are differences between DB vendors, but my question is about general patterns and best practices.

<comment><USER9> Chris Date's Introduction to SQL [1], or Edgar Codd's last book [2].

Actually both are worth reading to discuss differences in philosophy (e.g. "no NULLs" vs 4-valued logic including "unknown" and "not applicable" NULLs).

[1] [https://www.amazon.com/Introduction-SQL-Mastering- Relational...](https://www.amazon.com/Introduction-SQL-Mastering-Relational- Database/dp/0321305965)

[2] [https://www.amazon.com/Relational-Model-Database- Management-...](https://www.amazon.com/Relational-Model-Database-Management- Version/dp/0201141922/ref=sr_1_1)

<comment><USER10> First book [1] is not by C. J. Date. So, is this the book you recommend and just mixed up the author's name, or... ?

<comment><USER9> Good catch. Too late to edit the comment.

Yes, I meant C.J. Date's "Introduction to SQL".

I was trying to get a link to the latest edition which appeared to be the 2nd link on an Amazon search for "Date introduction to SQL"... anyway, it is continuously edited so it's worth seeking the latest version which has twice as many pages as some of the earlier ones.

<comment><USER10> The thing is, and I probably should have pointed it out in the first comment, sorry about that: I can't find such a book by Cris Date. My guess is you meant "Introduction to Database Systems".

<comment><USER9> Yes, you are right. Database Systems.

</comment></comment></comment></comment></comment></comment><comment><USER11> Could you write their example

        SELECT members.firstname || ' ' || members.lastname
        AS "Full Name"
        FROM borrowings
        INNER JOIN members
        ON members.memberid=borrowings.memberid
        INNER JOIN books
        ON books.bookid=borrowings.bookid
        WHERE borrowings.bookid IN (SELECT bookid
           FROM books
           WHERE stock>(SELECT avg(stock)
           FROM books))
        GROUP BY members.firstname, members.lastname;

more simply as

        SELECT @avgstock = avg(stock) from books;

        SELECT members.firstname || ' ' || members.lastname
        AS "Full Name"
        FROM borrowings, members, books
        WHERE members.memberid = borrowings.memberid
          AND books.bookid = borrowings.bookid
          AND borrowings.bookid IN
            (SELECT bookid
             FROM books
             WHERE stock > @avgstock))
        GROUP BY members.firstname, members.lastname;

Are those explicit INNER JOIN clauses necessary? By now, most SQL query optimizers can figure out on their own how to join tables.

<comment><USER12> My impression was that few people use the comma syntax these days, it's old fashioned, harder to read, harder to change, harder for the query plan optimizer, etc.

I know this is an appeal to authority, but the consensus is pretty clear if you search SO and programmer stackexchange:

[http://stackoverflow.com/questions/1018822/inner-join-on- vs-...](http://stackoverflow.com/questions/1018822/inner-join-on-vs-where- clause) [http://programmers.stackexchange.com/questions/78225/using-j...](http://programmers.stackexchange.com/questions/78225/using- join-keyword-or-not)

You're also introducing a lot more concepts by adding variables, and that code wouldn't work on SQL Server, for example, as you need to declare the variable first. Then again the `||` doesn't work on SQL Server either.

<comment><USER13> As an active DBA I would tell you that most of the time I would expect the comma join syntax to come from someone with grey hair, its an older format that once was standard.

I would also say that if you are suggesting style, you might ommit that INNER from the JOIN (if you are targeting sql server its redundant.)

<comment><USER14> I tend to mix and match the two. The comma join looks way more natural to me. One thing I do habitually which I don't see other people do much is use aliases.

I learnt SQL through SAS which is where the excessive aliases came from and it kind of stuck with me.

<comment><USER15> The "comma join" disconnects the join from the condition. It is hard to find the actual join condition between all the "where" conditions. By using JOIN ... ON .., you explicitly link tables with their join condition and also separate join conditions from other filter conditions on the "where".

<comment><USER16> _The "comma join" disconnects the join from the condition._

Yes, because it's more important to first know _what 's_ being joined, then traverse that list looking for _how_ it's being joined. You get the full name in the conditionals, so it's not like it's a mystery what they attach to.

But I have grey hair, so there you go.... :)

Comma syntax makes much more sense to me. But now I know that this is not The Way Things Are Done. I also learned from the comments that comma-first isn't recommended. So now my code will be more legible. It's all good.

<comment><USER15> I'm 45. I'm so old that I'm proficient using (+) to code outer joins in oracle :) Comma-join was the standard for me, and I had a hate-phase towards LEFT/RIGHT JOIN. But I must admin that once I've really tried the "new" join- on syntax it has great advantages that outweighs the pain of change.

<comment><USER16> Thanks. Good to hear!

I'm 51, and I'm a "full stack" programmer, which used to just be a regular programmer before everybody started overspecializing. When I learned programming, you met a guy, he told you what he wanted, and you made the computer do that. You didn't say "But I'm just the Java guy"

It wasn't that you were an expert in a dozen things, you just figured out whatever the heck you had to do.

I've kept with this philosophy and it's served me well.

So I dive in and out of SQL. I picked up my habits on SQL intensive projects in the 80s and 90s.

It'll be difficult for me to switch until I get into a spot where I'm using it heavily for a week or two. Hopefully I'll remember this thread then and give the new way a shot. Sounds freaky, but hell, I wasn't crazy about pointer arithmetic when I first heard about it either. :)

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> This is a very naiive and simple question:

I really love the idea of SQL and am looking for a project to use it. It seems like a great way to encapsulate most of the state of your program so you can make your code more decoupled. For crunching logs or other data-dumps it seems like a no brainier, but for everything else I don't really have a a sense for the performance characteristics. Like would it make sense to keep objects' data in an SQL table (like a pImpl) ? is going over the rows of SQL table instead of std::vectors a different order of magnitude performance wise (I assume with something like SQL-Lite I can keep the tables in memory and not on disk).

I guess when is it appropriate to use a database and when is it not?

EDIT:

For context - I'm working almost exclusively in C++ and am usually pretty performance sensitive. In my last job we did radar simulations with lot of entities (radar platforms, aircraft, ballistic missiles etc.) that just lived in STD containers. Decoupling state from the simulation was always a bit of a struggle. Would keeping something like that in an SQL database make sense?

Another example. At the moment I'm working on an art project and I need to keep track of vectors and polygons which will be later used by a ray tracer. I'd also like to keep the state decoupled from my program. Would a SQL database be a bottleneck?

<comment><USER13> It really depends on your performance characteristics as you mentioned, you can certainly make a database fast as hell but most of the time I would recommend SQL for when you need ACID characteristics(Atomicity, Consistency, Isolation, Durability), not performance per-se because while SQL can be insanely fast, if you do not understand the abstraction it can get slow real fast.

If you find yourself storing large tables of information that is vaguely relational and you value the consistency of your data over raw performance I would generally recommend SQL engines.

<comment><USER17> I guess i'm more envisioning something like a SQL container. So if you have multiple instances of a pImpl class you would keep the actual state- variables/implementation in a DB. When you want to say do some complicated look up (ex: which objects fit some criteria) you would express that operation in terms of SQL queries

</comment></comment></comment><comment><USER18> Is there any open-to-everyone SQL database on the net that I can query to try out some simple operations after reading this tutorial?

<comment><USER19> You can try [http://sqlfiddle.com/](http://sqlfiddle.com/), although I've not used it.

If you don't object to running one locally, SQLite Manager [1] is a good Firefox addon that lets you visually manage local SQLite databases.

[1] [https://addons.mozilla.org/en-US/firefox/addon/sqlite- manage...](https://addons.mozilla.org/en-US/firefox/addon/sqlite-manager/)

</comment></comment><comment><USER20> Anyone have a resource for mysql 5.6/5.7 about geoqueries and storing json? Also, resources about how people should build their table schema? It seems like what I've learn from school might be very different on how you should do it industry.

<comment><USER21> You'll need 5.7 to make full use of JSON features; ditto for geo. MySQL 5.7 is still considered pretty new (GA for ~9 months) so many companies haven't upgraded yet. Amazon RDS and Google Cloud SQL do offer 5.7 though, if you want to try out these features.

Best MySQL resources are generally the manual [0],[1] and Percona's blog [2],[3]

As for MySQL table schema best practices, definitely a broader topic, but yes it tends to differ greatly from what's taught academically. Baron's book [4] is probably the best starting place -- it's a few versions old by this point, but most of the core recommendations around InnoDB tables and indexing in there still apply.

[0] [http://dev.mysql.com/doc/refman/5.7/en/spatial- extensions.ht...](http://dev.mysql.com/doc/refman/5.7/en/spatial- extensions.html)

[1] [https://dev.mysql.com/doc/refman/5.7/en/json.html](https://dev.mysql.com/doc/refman/5.7/en/json.html)

[2] [https://www.percona.com/blog/2016/02/03/new-gis-features- in-...](https://www.percona.com/blog/2016/02/03/new-gis-features-in- mysql-5-7/)

[3] [https://www.percona.com/blog/2016/03/07/json-document- fast-l...](https://www.percona.com/blog/2016/03/07/json-document-fast-lookup- with-mysql-5-7/)

[4] [http://shop.oreilly.com/product/0636920022343.do](http://shop.oreilly.com/product/0636920022343.do)

</comment></comment><comment><USER22> Aside from using temp tables is there a way to assign sub-queries as a variable and refer to them later on?

completely simple and contrived for example purposes only, something like:

        active_users = select * from users where user.login > 0;

        select * from active_users where active_users.country is not null;

seems like that would clean up the query nesting quite a bit. Might need some kind of begin / end blocks like with transactions so the query planner would know the scope of those sub-queries.

<comment><USER23> Yep, a few.

1\. Temporary views. This one's my favourite but not everything supports it and sometimes DBMSs have weird limitations, not like allowing parameter binding in them.

2\. Temporary tables, as you mention

3\. CTEs ("with expressions"). These would be my favourite if they weren't an optimisation fence in postgres.

        WITH active_users
            AS (select * from users where user.login > 0)

        select * from active_users where active_users.country is not null;

4\. If you're only using it once, a subselect lets you add a name to it if that's all you're after

        select *
        from (select * from users where user.login > 0) AS active_users
        where active_users.country is not null;

5\. If you're using active_users frequently and want it to be globally accessible, a regular view. Often you can make a DB much more useable with a few well placed views like this and you'll need a few less abstractions as well.

<comment><USER22> Thanks for the options. can you expand on why CTEs are a postgres optimization fence?

as for 4: I am aware of the named subselect but once you add a few of those to the main select statement you start to get into unreadable territory.

back to CTEs, it seems like it should be an easy substitution problem for the postgres query planner. I am not a DB developer so I am probably not aware of all the issues. The CTE seems like the thing I was trying to describe though. Do you think that the optimizations would be more possible with begin and end scopes? It seems like they are part of the same select statement so i doubt scopes would help it.

<comment><USER24> In practice optimization fence is not normally a problem, and can even be an advantage. The general rule is - just use CTEs. And the specific rule is - learn to profile and optimize using EXPLAIN ANALYZE

</comment></comment></comment></comment><comment><USER25> Thank you for this. It may be too basic for most of the HN crowd but for me it is a good read for my further development.

</comment><comment><USER26> Every article that makes it clear that JOIN are effectively operating on tables and thus informally part of the FROM clause (rather than being independent clauses) deserves upvoting.

</comment><comment><USER27> The one thing I've always wanted to be able to do is to be able to control the projection part of the query. So in other words, I'd love to be able tell the select to display all attributes that are like '%name%'.

<comment><USER28> Have you tried using CASE with HAVING?

         SELECT
         ...,
         CASE WHEN attributes LIKE '%name%' THEN name
              ELSE null END AS name
         FROM table
         HAVING name IS NOT null

Or did you mean selecting by the column's name specifically?

<comment><USER29> I think GP means "SELECT all columns that have %foo% in the name of the column". It's not possible directly though you can do it with dynamic SQL by querying the data dictionary and dynamically creating the columns retrieved in the SELECT clause.

<comment><USER30> I think you can search for column names by querying the information_schema to the table schema (at least in postgres)

      select
        table_name
      from
        information_schema.columns
      where
        column_name ilike '%foo%';

see [https://www.postgresql.org/docs/devel/static/infoschema- colu...](https://www.postgresql.org/docs/devel/static/infoschema-columns.html)

For searching for items with '%foo%' in all tables and columns, yeah i think that you're probably right. [http://stackoverflow.com/questions/5350088/how- to-search-a-s...](http://stackoverflow.com/questions/5350088/how-to-search-a- specific-value-in-all-tables-postgresql)

<comment><USER31> > "For searching for items with '%foo%' in all tables and columns"

To do this you could perform two queries of the information_schema (one query for '%foo%' in tables, one query for '%foo%' in columns) and join them together with a UNION statement (or if you didn't mind duplicates you could use UNION ALL in this case as it's faster and the results from both queries can have the same fields).

</comment></comment></comment></comment></comment><comment><USER32> I love this guide and will probably steal ideas from it when writing my own SQL tutorials. That said, I don't think SQL is _that_ hard to read compared to other computer languages, at least for non-programmers.

I say this as someone who learned object-oriented programming far before I learned about SQL or its declarative model. SQL was "hard" for me because the declarative style is so functionally different than OOP, and to this day I still haven't ventured to figure out the SQL equivalents of functions, variables, and other mainstays of general programming; I'm satisfied with using a scripting glue language like Python to automate simple SQL SELECT statements.

But I love teaching SQL to non-programmers. You can do a lot of important real-world things with it without having to jump deep into the logical puzzles of "regular" programming. By "deep", I mean things like for-loops; it never fails to surprise me how befuddled beginners are about how for-loops work -- not just the syntax, but the concept of a "block" of code, that for-loops can exist inside of for-loops...or more fundamentally, you can run more than one line of code inside a for-loop.

With SQL, you don't have to worry about any of that. You simply declare what columns you want from a table, what values to filter by, and optionally, how to aggregate it or join it with another table -- concepts that have much clearer analogies to everyday thinking about information (or, at least spreadsheets).

SQL gets complicated when you try to do complicated things. In the OP's opening query, there's a joining of three tables. But without reading it closely, I can already kind of describe the real-world information-need that it fulfills: List the people who have borrowed books that have been borrowed _something something_ to do with `stock`. If I spent a few more minutes, I could figure out what the fuck that subquery, `stock > (SELECT avg(stock) FROM books)` is all about, but the difficulty there is just my ignorance of the domain and/or particular conventions in this hypothetical problem. If I, or even the average novice, wrote this query line-by-line, its purpose would not be so opaque (as writings rarely are to their own authors). The point is that it's much easier for a beginner to author their own SQL without fucking things up in the way you can easily do so with even just a for-loop. I can't imagine that the Python or Ruby equivalent would be nearly as clear as this SQL statement, even for a far simpler kind of join.

My main complaint about SQL is that it seems more logical that the `FROM` clause should come before the `SELECT` statement. Though my initial guess is that that hampers the declarative-clarity of the statement when `JOINS` are involved.

edit: If I could make one suggestion to the submitted article, it would be that the first "Simple Query" is not simple at all:

         SELECT bookid AS "id", title
         FROM books
         WHERE author='Dan Brown';

I'd argue that this is a simpler statement, and one in which the results are immediately recognizable to every spreadsheet user:

         SELECT * FROM books;

The default behavior of Excel is to show all columns of a data file. When you get more advanced as a SQL user (and more advanced in understanding the fundamentals of computation), _then_ you start being selective about SELECT and realizing that, as an end user (nevermind the cost of computation), it's not always ideal to see everything by default:

         SELECT title FROM books;

<comment><USER33> Ironically most programmers I know have no trouble with loops but struggle with SQL. The most common SQL error they make is trying to use loops. (RBAR: Row by Agonizing Row)

<comment><USER31> If a programmer is struggling with SQL in the way you describe, I'd suggest they learn how to perform MapReduce in the programming language of their choice. That should help give them a better understanding of how to approach SQL. If their language doesn't have good support for the MapReduce approach, a quick introduction in any functional language should help.

<comment><USER32> I teach SQL to students who may have no programming experience at all, unfortunately

<comment><USER31> The MapReduce advice was only meant for programmers who had to unlearn imperative programming habits. Non-programmers won't have imperative programming habits to unlearn.

</comment></comment></comment></comment></comment><comment><USER34> Some of the tables don't work in Safari, but otherwise a good intro.</comment>


Ask HN: Any PHP devs interested in working on Squerch? - <AUTHOR>

My name is David and I am a London-based interaction designer in my day-job, but I have been working on a startup idea for a while now, and would really like to take it to a more serious stage, hopefully getting onto a program like YCombinator. I launched my startup - http://squerch.com - a couple of months back to get some initial user feedback, which has so far been good, even though the algorithm isn't performing as I'd liked. A programmer friend of mine has been doing the PHP coding up until now, but he is so busy with the day-job, it's slowing down the development of the site, when I want to push it forward. I really <i>really</i> want to make a success of the business, but at the moment I'm pretty much a one-man band, and need to build a team to take it forward. As I have no funding, I will be looking for a developer who sees real value in the concept, and will take good equity in the business. Work on the project will be part-time to start with, but will be growing with every tweak of the search algorithm ;) Loads more ideas planned, but need a dev to help it grow quicker. Get in touch :) <comment><USER2> What's preventing you from working on the algorithm yourself? What will you be contributing to the effort if someone takes over that work from your friend?

<comment><AUTHOR> I am a designer, so will be carrying out the (HTML, CSS, JS) changes to UI, whilst carrying out more basic day-to-day PHP stuff as well. Plus I will be concentrating on the business side of the venture. Are you interested in the project?</comment></comment>


Google Taking Aim at Device Modders in Android 4.4 KitKat - <AUTHOR> http://www.xda-developers.com/android/google-taking-aim-at-device-modders-in-android-4-4-kitkat/ <comment><USER2> Just as a hypothetical example, let's say I want to hook up a Wiimote to an Android device.

The Wiimote cannot enter a code and hit enter, but apart from that it's a normal Bluetooth device. Pre-Android 4.2, it was perfectly fine to connect it. Bluetooth uses the code '000000' for this and a third party application can hook through.

With Android 4.2 and 4.3, the ability to connect Bluetooth devices to connect without using a code determined by the phone was removed. It was possible to root the phone and set it to a specific MAC address for the Bluetooth interface, from which the Bluetooth code was generated, so that this Bluetooth code would be '000000' and the Wiimote would connect correctly.

Now with Android 4.4 the answer is going to effectively be 'forget it'...

I'm sorry but if what seems to become the most ubiquitous operating system out there cannot connect to probably the most ubiquitous gamepad in recent memory (and quite a few others that exhibit the same problem), using a standard that both of these devices support, then that's just a tad ridiculous. Especially if it's something as simple as this that was supported in the past.

And that's just one example.

Sure, I'm also going to complain about the fact that "it's my device, I want to use it as I want". But when you actually see the reasons coming through behind this you might understand why rooting might not be such an unreasonable request - it makes things like that, things that should work, work.

<comment><USER3> Right now there's two ways to root some devices -- though the bootloader (unlock), which allows you to securely do the changeover. And root exploits, which are security holes in the operating system and can be used to persistently write insecure/root functionality on the filesystem.

The former can only be done by you (the device owner), but in theory the latter can be done by any manner of malicious applications. dm-verity only addresses the latter.

You should still be free to unlock and flash your own ROM to your hearts content. This includes flashing rooted images that have whatever bluetooth functionality.

<comment><USER4> Of course, the reason root exploits are so popular as a way of unlocking devices is because most Android devices don't have unlockable bootloaders. I mean, I guess technically Google aren't stopping people from unlocking their devices, they're just giving the device manufacturers and phone networks a more robust way of doing so, but that doesn't really make much difference to the end user...

Also, at least on ChromeOS unlocking the bootloader disables playback of DRMed content.

<comment><USER2> Unlocking the bootloader nominally both voids the warranty and requires a complete wipe of everything on the phone.

<comment><USER3> Requiring a wipe is _by design_. If an attacker steals your phone, whats to prevent them from flashing a new system partition that lets them read off all the existing data? You can backup most (all?) of your data manually, then unlock and restore the data. Otherwise unlocking would be essentially the easiest attack vector to owning a device without the user knowing.

</comment></comment></comment></comment></comment><comment><USER5> If people are modding their phone by loading a new kernel as well as the modified file system, they won't have a problem. So for people with unlocked bootloaders, this isn't a problem; and if a handset doesn't allow the bootloader to be unlocked, refuse to buy it! There are plenty of perfectly good (and in my opinion, superior) choices out there which allow for the bootloader to be unlocked. All of the Nexus devices, for example....

There is another advantage of using a locked down file system --- you may not be able to use a rootkit to break root and then modify the file system, but some unfriendly adversary, such as the FBI or the NSA, won't be able to do it to your phone without your knowledge, either. I would think that's worth the inconvenience of making sure you buy a phone with an unlockable bootloader. (Since unlocking the phone requires a wipe of the disk, you don't have to worry about the FBI/NSA doing this to access your data on the phone without your knowledge.)

<comment><USER6> The nexus devices do not support verzion's lte network, so in the end you are left paying 650-900 for a developer version. That's fine for me, I'm a mobile developer but not many who aren't directly making money off of the device would be willing to make the same choice.

<comment><USER7> _you are left paying 650-900 for a developer version_

The Nexus 5 is $350-$400. The Nexus 4 was even cheaper. I got mine for $200 when the sold off their remaining inventory.

Also most of us don't _need_ Verizon.

<comment><USER8> He means to get 4g on Verizon w an unlocked boot loader you end up paying for the dev edition.

<comment><USER9> That's incorrect though. The Verizon Galaxy Nexus has an unlocked bootloader and works fine on their LTE network.

</comment></comment></comment></comment></comment><comment><USER10> It seems every headline could be interpreted in 2 ways, and I've seen a bunch of them lately:

1\. a) "Evil Google Tries to Take Control of Android"

       b) "Google Tries to Standardize Android and Fix Fragmentation"

2\. a) "Google Taking Aim at Device Modders in Android 4.4 KitKat"

       b) "Google Trying to Make Android as Secure and Unhackable as ChromeOS"

Doesn't it make sense for Google to try and stop rooting through vulnerabilities, just like Apple tries to stop jailbreaking with every update?

From what I understand, rooting should still be possible if the bootloader is unlockable (which it is for Nexus devices, HTC's devices, and I think Sony's too). Samsung and a few others, on the other hand, don't let you unlock the bootloader. So maybe we should take it up to them, and demand unlockable bootloaders, so we can modify our devices, instead of asking Google to turn a blind eye to Android exploits.

<comment> k-mcgrady >> "Doesn't it make sense for Google to try and stop rooting through vulnerabilities, just like Apple tries to stop jailbreaking with every update?"

The problem is that Apple doesn't position iOS as 'open'. Google does. That's been one of their big selling points for years.

<comment><USER11> Google positioning its OS as "open" doesn't mean they have to make it easier for script kiddies and hackers to find vulnerabilities in your phone and exploit them. The fact that it makes it harder for modders to (legitimately) root some (some!) phones is unfortunate, but I think is just going to have to be a casualty in the name of better device security.

As another poster said, we should be taking this up with the Samsungs who ship devices with non-unlockable bootloaders.

<comment><USER12> Being _open_ in this context is like being _pregnant_ , you either are or not. You can't be _half-pregnant_ or half-open.

The remark about the script-kiddie able to _find_ vulnerabilities is ridiculous for a wide variety of reasons. Script-kiddies are not able to find vulnerabilities in any kind of system. Computer hackers obsessed with security do that and (most of the times) it's not easy by any means, i.e. Saurik.

<comment><USER13> Firstly, you meant "e.g.", not "i.e."

In addition, I'm not sure what Saurik, the creator of Cydia, has to do with this discussion at all.

Finally, and most importantly, you don't seem to understand the current mobile market. Most users are the end-all-be-all of end users. Many people I know think that most of the apps on the iOS App Store are developed by Apple. People install apps without thinking. Tons of personal information is stored on your phone and in these apps, such as contacts and banking information.

Google/Android is open in the sense that Android is open source, so hypothetically anyone can hack on it.

 _You_ are the person who is half-pregnant. You want the normal end user experience, where you don't have to worry about compiling the operating system yourself, but you want it to be "hackable". That is not the definition of open. That is the definition of insecure. Because most Android users are not sophisticated computer users like yourself, a hackable operating system is just a bad idea.

If you want an OS that you can do whatever you want with, and is incredibly insecure, then fork Android and have at it. Do that, instead of complaining whenever Google makes changes that you don't understand and affect your somewhat specific and not very common use case.

<comment><AUTHOR> >In addition, I'm not sure what Saurik, the creator of Cydia, has to do with this discussion at all.

He has found various security bugs in Android (most recent example: [http://www.saurik.com/id/19](http://www.saurik.com/id/19) from the other day) and he is also the developer of [http://www.cydiaimpactor.com](http://www.cydiaimpactor.com)

atmosx's point was that it isn't 'skiddies' who are finding these security holes, it's experienced developers and security researchers.

</comment></comment></comment></comment></comment></comment><comment><USER3> There seems to be a lot of confusion and misinformation about what dm-verity actually does in this thread. It's the same verified boot mechanism that's used by ChromeOS[1].

This just guarantees that on a locked, signed device (i.e. you haven't unlocked it to flash custom ROMs), the filesystem there is the one that's supposed to be there. Things that would be caught by this are (as the original documentation says[2]) rootkits and other persistent exploits.

You should still be able to unlock and flash your device; this is supposed to make it harder for a malicious app to root and own your (locked & signed) device persistently without you knowing.

[1] [http://www.chromium.org/chromium-os/chromiumos-design- docs/v...](http://www.chromium.org/chromium-os/chromiumos-design- docs/verified-boot)

[2] [http://source.android.com/devices/tech/security/dm- verity.ht...](http://source.android.com/devices/tech/security/dm-verity.html)

<comment><USER14> If I cannot unlock my device, modify its software, and then relock it and continue to use the modified software, that is serious issue: you should not run your Devi e normally with an unlocked bootloader as there is then no protection against someone picking up your device, booting it into fastboot, and flashing new software (which normally is protected against as the unlock process erases all your data). This does not actually seem to verify things at this level, though (and thereby does not seem similar to the ChromeOS mechanism).

</comment></comment><comment><USER15> My standard procedure when I get a new android device is to (a) unlock the bootloader, and (b) install su & SuperSU, i.e. get root.

Normally the latter involves modifying the base filesystem. If I understand it right, this will prevent the normal rooting procedure and force the need to install a different kernel just to get root. Which isn't great, as I have no desire to run a non-stock kernel; they come out later with stable builds, they may have issues finding all the right drivers for the phone, etc.

I may be mistaken, and if root can still be gotten without switching the kernel, then everything is OK. Otherwise, this is a big dent in the desireability of Android to me; root is incredibly useful for doing things outside the constrained sandbox, like mounting encrypted file systems, or syncing the clock to an NTP server (!).

<comment><USER16> Well if the manufacturer isn't violating the GPL, you should be able to easily rebuild the stock kernel without this verification. But of course, that's a big 'if'.

I'd still love to see a significant kernel contributor step up to the plate and force the issue that locked bootloaders violate even GPL2. Of course all of these issues would be more straightforward if Linux had simply moved to GPL3 while it still could have.

</comment></comment><comment><USER17> Tl;dr "To re-iterate, if you are able to change the kernel your device uses, this feature will not be a concern. It's possible to either disable dm-verity in the kernel, or to set it up to use your own keys to authenticate the system hash. For users who choose to buy carrier-branded devices and accept a locked bootloader, but find a way to root the device, take heed of this warning. It's not at all unlikely (in my technical opinion) for this to happen on future devices. If you want the ability to modify the software on your phone, I'd avoid anything with a locked bootloader, and ensure you can modify the kernel (to disable or modify the dm-verity signatures)."

<comment><USER18> So just 90% of all Android phones sold.

<comment><USER19> 90% of people don't want root exploits on their phone either. Let's hope it's the same 90%!

</comment></comment></comment><comment><USER20> There's great amount of very talented people hacking on Android devices. The best thing we can hope for is that Google makes Android so anti-consumer, anti-developer and anti-hacker, that they all move Firefox OS or some other mobile GNU/Linux variant that's not Android.

This goes for manufacturers as well.

<comment><USER21> That's unlikely when so many people in the tech sphere (PG included) paint Google as an 'open' benevolent company fighting the 'evil' of everyone else.

<comment><USER22> That seems to have changed over the last few years.

<comment><USER21> A little, but they are still regarded as 'better than the others' by those who for some reason desire that Google obtain a monopoly over search _and_ the OS.

</comment></comment></comment></comment><comment><USER23> Excluding Nexus devices, which manufacturers are currently hacker-friendly (i.e. let you to unlock the bootloader without exploits)?

<comment><USER24> Sony let me unlock my Xperia Z. They also release the drivers to their hardware to the open source community. Well worth supporting this effort when you next choose a handset.

[http://unlockbootloader.sonymobile.com/](http://unlockbootloader.sonymobile.com/)

</comment></comment><comment><USER25> People really need to remind themselves that rooting Android devices is fairly stupid if you actually intend on using them for anything. You're essentially turning off any of the security on the platform.

It's also separate from the question of being able to install your own OS build, which is of far greater importance.

<comment><USER26> That would be a fine argument if the "security" of the platform did not include _attempts to stop people from doing normal, useful things_. Why should I not be allowed to tether my phone, when the technology is there and I am already paying outrageous amounts for my data plan? Why should I have to pay my carrier _even more_ just to have software features enabled? Why should I not be allowed to use my phone on another network?

See, "security" in this context is really about securing the carrier _from_ users and those who might threaten users.

</comment></comment><comment><USER27> I love the Android dev community so I'm sad to see that the job is getting harder, but I don't think that implementing a verified boot mechanism is necessarily an anti-modder move by Google. They've implemented security like this to great effect on Chrome OS, which is one of the most secure operating systems you can get out of the box. It would be great for the __average __person to know that their Android phone is super-secure, and it would encourage the penetration of Android into enterprise and military applications, where security is a huge concern.

</comment><comment><USER28> When I read the heading I thought, cool more things to make modders' lives easier.

Boy was I wrong :(

<comment><USER29> In general the term "Taking Aim" refers to putting something in your target and shooting it... i.e. killing it.</comment></comment>


Snapchat has a huge problem with Android, and it's causing investors to worry - <AUTHOR> http://uk.businessinsider.com/snapchat-huge-problem-with-android-causing-investors-to-worry-2017-2?r=US&IR=T <comment><USER2> Snapchat is based on a lie. You can't give someone information and also control what they do with it, to make sure they delete it. It's the very definition of a passing fad, it doesn't do anything unique or special not done many times better by any other instant messaging program. It makes sense then that its growth would slow, because after you realize you can't rely on its singular selling point (the ephemerality of messages you send to other people), what's left is a crappy version of any other program that lets you share pictures with people.</comment>


It's time to take Dogecoin seriously - <AUTHOR> https://medium.com/bidali/its-time-to-take-dogecoin-seriously-b8a52c916534 <comment><USER2> I keep saying that. DOGE has many advantages that are hard to find in other coins at the moment.</comment>


Ask HN: How does your team handle knowledge documentation? - <AUTHOR> We are currently going through a rethink of our documentation (not for customers, just our internal processes, tools, etc.). Currently we have a docs directory in the root of our project repository which holds a bunch of markdown files, but we are beginning to grow out of this. Keeping links accurate and update to date isn't too much of a challenge, it's the barrier to entry. Filing a PR to make a minor documentation change is just too much. We could have the documentation in a separate repository, but if we are going to be making this leap, we want to be sure we are using the right tools for the jobs.
The way we see it, we are going to end up requiring at least 2 different types of documentation. The first is our deep documentation of our tools. This explains exactly what it is, how it works, etc. and is designed for people who want to work on these sorts of tools, not with them. The second is a quick fire Q&A. "I'm having issue X, how do I fix it?" Think StackOverflow.
So, what does your team do for these challenges? How effective is your solution? What do you think would be better? <comment><USER4> I work at a large bank. Mostly people hold all relevant knowledge in their own head, and when they receive a request for information they only respond if their manager knows yours. Then, they will mostly refuse to create any type of actual document and instead request that you set up a meeting with them through Outlook. They are of course completely booked on their Outlook calendar for at least the next few weeks, and then when you do get to talk to them they use obtuse department-specific acronyms as much as possible so that by the time you've started to understand the surface of their answer they have to leave for an urgent meeting.

<comment><USER5> I thought I didn't work at a large bank but you're making me doubt that, we must be in the same building.

</comment></comment><comment><USER6> Solution: Hire a librarian. I'm not kidding in any way. They are massively underemployed and are very good at exactly this task. Back at PBwiki we hired a librarian who not only organized all the things but ended up running and building our support organization.

Do not tell them what tool to use, let them own your knowledge base and make their requests for information understood to be P1 priority.

<comment><USER7> I am afraid that the idea of hiring an expert for a specialized task goes against the trend of "full-stack developers".

Imagine explaining why it is not necessary for the project to have a database expert, a back-end expert, or a web expert... but you definitely need a wiki- pages-organizing expert.

<comment><USER8> Perhaps that's a cue that "full stack developer" is just a cute buzzword for "jack of all trades, master of none".

</comment></comment></comment><comment><USER9> At Vistaprint, we used MediaWiki, hired a dedicated librarian, and, most importantly, had a culture of documenting everything on the wiki. The last point is most important. Everyone in the organization _must_ adopt the mindset of checking the wiki first. If the information is not on the wiki, figure it out, and put on the wiki. I vaguely remember wiki articles even being a (minor) metric for annual reviews. At one point, Vistaprint tried SharePoint but we quickly went back to MediaWiki.

At edX we have information spread between Confluence and Google Drive. Search is not ideal for either. Information sharing is not great.

</comment><comment><USER10> Disclosure: I am the founder of a company that aims to solve this first documentation case you pose. But the process of validating this problem and get early feedback on our solution, I interviewed to dozens of companies to learn about their tools and processes, and hopefully some of that can be helpful here.

The information split is very much as you describe between canonical and ephemeral. For the first case, the defacto tool for medium size companies (25-1000) is Confluence (large companies lean towards a custom intranet). Confluence is accessible to the entire company, not just engineers with git and markdown knowhow, it is also well established, very reasonably priced and can be deployed on cloud or on premise. There are drawbacks such poor search and complex organization that makes it hard to find anything, but there is not a clearly better solution at the moment, though there are a few startups like mine trying to change that.

For the second case, companies have tried to deploy an internal StackOverflow / Quora, some homegrown or using open source tool but unless tied to a specific workflow (like all hands Q/A), they are eventually abandoned. The issue is a lot of duplication of content that also changes very often. So long term storage is not as valuable as just removing the initial friction and what seems to work best is an ephemeral solution like a dedicated Slack/Hipchat/Teams/Mattermost/Gitter/Zulip channel.

<comment><USER11> what is the company called you founded?

<comment><USER10> The company is called Slab (somewhat of a double reference to a thing you can write on and slab serif fonts). It is current in private beta but if you would like to take an early look please find my email in my profile and I would be happy to share with HN.

<comment><USER12> How is your solution different?

<comment><USER10> We is much simpler and easy to use and plays well with the non-Atlasssian ecosystem (as well as with JIRA). In both these regards we like to think of ourselves to Confluence as Slack is to Hipchat.

</comment></comment></comment></comment></comment><comment><USER13> The PR isn't your problem. There will be some other other problem you'll find with any new documentation system. The root cause is keeping documentation up to date is universally a pain. Unless you have it as (part of) someones job, it always will be subject to decay.

If someone has "solved" creating a good way to capture tribal knowledge, I'm all ears. But it's not the tools being used that are the issue or the solution.

<comment><USER14> Our company has a culture of making it a part of the dev owner (or product owner's) job. So as part of shipping a feature / epic / task you are expected to create and update documentation. The added benefit is when someone has a question you point them to the document and make sure they read it before scheduling a meeting.

</comment></comment><comment><USER15> In my opinion you are on the right track with the project holding a Documentation folder and the markdown files. Keep it all together or you will have yet another place to maintain.

I would suggest adding the use of Doxygen to your projects. It supports Markdown files as well as Markdown syntax in your code comments. You might find that a lot of what you are documenting in the separate .md files could be a notes or remarks section of your method class comments. This will remove the separation of technical documentation vs. help/FAQ comments by doing all of the documentation in the code. You can then add Doxygen to your build, so that building the project refreshes the documentation. Doxygen can be configured to produce a online html version that can be deployed to an internal web server for quick reference. It can also build .pdf, rich text documents, and unix man pages.

Then if you use Continuous integration/deployment, when you deploy to test, UAT, then eventually production you deploy the official documentation that was generated.

The challenge with all documentation, is, Actually doing it. So it should be part of your SDLC documentation and process. the project/task/request is not done until documentation updated. Make it a checklist item that needs to be reviewed in code reviews. Doing the technical and help/FAQ all in code comments makes that easy.

cheers!

<comment><AUTHOR> We already have this, the problem is more for processes. Things like "How do I create a build from my commit?". We have dedicated documentation which explains how the build system works in it's entirety as markdown files, but it's painful for a new team member to have to go through that and find out what they need.

A simple Q&A document is failing us as well due to the fact that all commits to our main branches need to go through PR which puts in a high barrier to entry for what might be a minor change. It just discourages collaboration in so many ways.

<comment><USER16> Maybe the problem can be reframed?

> the problem is more for processes. Things like "How do I create a build from > my commit?"

With well set up CI and automated builds, would you need this documentation at all? Could other processes which you currently document also be automated?

In my (limited) experience there's no great way to handle documentation, apart from reducing the need for it as much as possible.

The team I'm on (as a PM, I'm not a programmer) used to use Confluence for almost all technical documentation, apart from the odd Readme file. We use it for product, product range and other team documentation too, which makes choosing sensible top level hierarchy difficult! I don't recommend trying this.

But we finally have decent automated builds in place, and it's made a world of difference. We will soon have decent CI set up too (with Gitlab), at which point lots of project specific technical documentation will move to Gitlab wikis, living alongside the repo it relates to.

<comment><AUTHOR> > With well set up CI and automated builds, would you need this documentation > at all?

We have that. But how would you know that we have that without documentation telling you? How would you know where to find the resulting builds or to check the status?

<comment><USER16> Fair enough. We currently document such systems details in Confluence, which I can't recommend for all the reasons given by others here.

> We could have the documentation in a separate repository, but if we are > going to be making this leap, we want to be sure we are using the right > tools for the jobs.

If you haven't considered this yet, it would be very easy to set up a Mkdocs[1] site for testing, using all of your existing documentation. Should be as simple as making a new site and dragging your current docs folder contents into the docs dir. It has good search built in with Lunr[2]. Try the Material theme[3] if looks are important.

[1] [http://www.mkdocs.org](http://www.mkdocs.org) [2] [https://lunrjs.com](https://lunrjs.com) [3] [https://squidfunk.github.io/mkdocs- material/](https://squidfunk.github.io/mkdocs-material/)

</comment></comment></comment></comment></comment><comment><USER14> Our company uses Confluence and I highly recommend it. I think it does a very good for keeping your documentation "fresh" because:

* It's easy to search and find things

* It's easy to be notified of updates

* It's easy to comment & edit (for both non technical and technical members)

* It's easy to create quick documentation through the Q&A product and use the wiki for longer documentation

The main cons I've found with Confluence is:

* The general UX is weak (this is the case across all Atlassian products in my opinion)

* The markup language is not markdown

* The edit & save UX is a bit too heavy, your changes don't get saved automatically but rather it requires a separate button change. So documentation changes are a bit slower than something like Google Docs

Despite these cons, Confluence has been great for us. However, in order for any tool to work there also has to be a cultural shift. Our company is very good about:

1) Keeping knowledge not silo'd in brains of devs or other tools. So whenever I ask any senior dev a basic question like "How do I get unit test coverage?" they will immediately link you to a confluence Q&A answer or if one isn't created they will give you the solution and immediately ask you to create a confluence question.

2) Not spreading knowledge across other tools - any process, decision, design, etc. is required to be in Confluence. Any thing in Slack, Email, or Google Drive is immediately asked to be moved into Confluence.

</comment><comment><USER17> I work in a department that develops and maintains an internal SDK used by internal BIUs to develop medical applications. Size of codebase is well over 1 Million LoC.

We heavily user source code documentation for all public API which is enforced by our internal build tools (build breaks when undocumented code is found) and the review process (to check if documentation is proper). We have a searchable themed MkDocs page with a large number of "how to properly document X". Reviewers actively place links to these pages in the reviews where source code documentation is not proper (style issues, lacking, no examples, etc).

Goal is here to have a proper MSDN like source code documentation and site generated by Doxygen. Each release gets its own documentation, which is easy as it is just a source control revision. A second goal is to have most of the documentation as close to the code as possible so it actually stays well updated.

Next to this we have a developer portal which is based on manually written markdown files. We are currently in the process of replacing our existing media wiki pages into markdown as well. Markdown is quite powerful as most people can easily read it, even with just notepad. In addition it converts to anything so it feels a bit more future proof. Right now we put it in a slightly extended mkdocs served site. We extended a bit for versioning and a landing page that links to separate mkdocs sites as the navigation becomes too cluttered if we put all of our layers and components in one site. Better to branch Early is what we found. So separate sites for database, infrastructure, system (multi-subsystem deployments), application development (wpf, HTML5,..),and for each subsystem (printing, auditing, reporting, data server, ...).

The problem is already we notice the manual markdown is not properly maintained. I think this is a problem you never solve unless you hire people to do it full time. I.e technical writers.

My personal opinion is that a stack overflow enterprise (on-premis SO) is more powerful. Nowadays people want to know "how" instead of "what". A Q&A site with tons of questions and answers is much better than a one-way written developer portal. Also, other people outside of our department could also answer questions for us. As a benefit you learn where people struggle with your API or use it for things it was not developed for. Feedback is great here.

<comment><USER18> Do you mind to share what kind of X in "how to properly document X" pages you have? I like that idea and would like to do similar!

<comment><USER17> I wouldn't mind but our bureaucracy organ does :(. Whilst technically it is only moving the markdown files and CIconfig from our private gitlab to a public one, the bureaucracy involved with it prevents me from doing it today. I would have to pull a part of our source code which is under strict regulation, and I try to only start that process when I absolutely must do so (subsidiary projects mostly).

The contents are free to describe though. Essentially you would find a large number of pages on all language concepts for our main languages. So "how to document a method", "how to document a property", classes, namespaces, you get the idea.

Next, we have a very clear setup in our documentation pages. All pages and sections on pages have permalinks. The idea is that you can redirect people to the exact information they have to read. Every section always starts with three examples. The examples are different enough to be applicable to say 80% of the cases. This means most of time a developer does not need to read more than the examples, c/p one and adjust to his specific case.

After the example cases the different aspects are discussed that should be in. This includes rules like "use the 3rd person when writing descriptions", "use present tense", "start with active verb" but also "include minimally one example in the format above" and "never use the name of X in the description". (I.e SetSomeCrypticThingOnlyWeUnderstand() // sets the cryptic thing << not helpful). Basically a checklist. Typically we have DO, Don't, and AVOID listings.

After that (bullet/checklist) usually a paragraph describing the rationale for documenting like this as well as other information not captured in the examples or bullets.

Sometimes after this there is a section of "also consider" where we capture issues of the past. E.g "when documenting X that uses units, specify the units.". I.e if you have a property Angle document it is degrees or radians. That same page would probably have a paragraph on should you name "Angle" or AngleInRadians or some info on avoiding ambiguous naming of properties.

I'd say the biggest game changers are the permalinks, good golden samples at the start of each section and in some cases bullet lists. The description paragraph is optional.

All our pages are markdown pages served on Gitlab Pages (on premise deployment) as static site built with MkDocs and Material theme (slightly customized) [1]. All pages include an "edit on gitlab" button so users can edit the pages in their browser without downloading any tooling. A CI setup will auto redeploy on commit/push. If clone the repo and you download MkDocs locally you can of course write with live reload etc on a local server (MkDocs serve).

O, and have a great search functionality and decent responsive website that is easy to read (good font, colors, syntax highlighting etc).

[1] [https://squidfunk.github.io/mkdocs- material/](https://squidfunk.github.io/mkdocs-material/)

<comment><USER18> What a great informative reply, really.

      Thank you.

I will definitely start to create a similar meta-documentation for our department as well.

Really how can we expect anyone to properly document anything when not specifying how? It is so obvious now.

</comment></comment></comment></comment><comment><USER12> We use mediawiki and and recently IT invested in confluence.

I hate it, because it's slow.

but I do see it has some benefits, for example, it automatically generates a content table. so content discovery is easier. it is also designed to be a document system for a software company. you can use it to create meeting notes and tag people.

but overall, I liked mediawiki, the problem with it is, it's hard to make sure things are up-to-date. and it's difficult to do content discovery.

<comment><USER19> We too use Confluence and it's abundantly unbelievably awful.

* Super bloated and laggy

* Hard to get your data out/migrate to some rival system. Did your technical writers spend a bunch of time typing strings into a table using the Confluence table building GUI, and now you want that data as a .csv? Too bad, hope you want to spend half an hour in copy-and-paste + reformatting hell.

* Impressively broken search (e.g. if you create a page named "foo" and then search for "foo," you'll often get pages of unrelated results that _do not include your page called foo_ )

* Can't natively show typeset math. Doing so requires plugins that are somehow both expensive, broken, and only compatible with narrow ranges of Confluence versions.

* Gross nonstandard markdown dialect

These are just the shortcomings that I can think of right now. However I think even Confluence is probably better than submitting PR's for docs changes as the OP mentions, since at least it's a wiki.

<comment><AUTHOR> These are basically the same issues we have. We've ruled it out entirely as a suggestion. Markdown documents and searching on disk is better than Confluence.

</comment></comment></comment><comment><USER20> How? Badly.

Right now it's a GitHub wiki in our "infrastructure" repo (which is basically all the bits and pieces that don't fit anywhere and don't have a proper repo of their own). We migrated to this from: "several github wikis, each in the repo of the project" \- but often you have overlap and it was bad to find stuff.

Pros:

\- easy backup (git clone)

\- markdown

\- easy to get started, "good enough"

Meh:

\- searchable only with a browser extension

Contra:

\- some coworkers don't like the github wiki

My dream (never happened in any company as of now) would be a single document (think: the manual), hopefully editable in a proper way (I am not sure if checking in changes in a doc/ folder in a code repo is the way to go for <10-20 people) that's hopefully displayable in a browser (and not a PDF, and ESPECIALLY not a Google doc) with cross-references.

<comment><AUTHOR> Good to know that Github wikis aren't searchable. We have a Github Enterprise installation and were considering using the wiki for our longer, detailed documentation, but lack of search is a no go I think.

<comment><USER21> You __can __search on Github wikis. You enter your query in the search bar to the right of the Github logo. I guess wink hasn 't tried to in the past year ([https://github.com/blog/2219-search-wiki- pages](https://github.com/blog/2219-search-wiki-pages)).

<comment><USER20> Wow, now that's undiscoverable. Thank you, missed that one completeley

Blog post is from August 2016 "You can now search for wiki pages from either the main search page, or within a repository."

Guess I made up my mind about that 1-3 years earlier and have used the extension ever since :) Ah well, one click less at least.

</comment></comment></comment></comment><comment><USER22> Don't use confluence :( In every team I've been, confluence feels like an huge hole, we can never find what we are looking for...

<comment><USER23> That's probably because you've used Confluence. If you used something else, then that'd have been the huge hole.

Confluence is a very good wiki. There's no wiki that creates good structures for you automagically.

<comment><AUTHOR> We actually used Confluence but moved away from it. The interface was difficult to use, it was slow, the search wasn't great, etc.

<comment><USER22> yeah, same pains that I've felt. I get the feeling that Atlassian products always lack some UI love

</comment></comment></comment></comment><comment><USER24> I have had great success with Mediawiki and Slate. With Mediawiki, you can use Categories, ex: [[Category:NameHere]], and templates (don't forget to update the CSS) for great organization and consistency. The usage of templates will allow you to be DRY, so if you update the template, where it is embedded will update too.

And Slate was just a pleasure to work with.

<comment><USER17> FWIW we are moving out of MediaWiki after 10 years. The search function is horrible and the readability on any devices is not great either (understatement). The latter is with our theme, not sure if you can theme properly. Long sentences, no proper image positioning and scaling, mobile support is lacking, etc. Also versioning is a problem when you need both versions of the page to be readable (e.g. 1.0 and 1.1).

It is great for one thing, everyone can edit it with just their browser. Today, however, you have more options that can do that with a Git+CI backed auto deploy on change.

<comment><USER25> Did you ever try Elasticsearch on your MediaWiki instance?

</comment></comment></comment><comment><USER26> If you want to maintain documentation for different versions of the same software then consider AsciiBinder. [http://asciibinder.org/](http://asciibinder.org/)

For example, Openshift Origin documentation uses it: [https://docs.openshift.org/3.6/architecture/index.html](https://docs.openshift.org/3.6/architecture/index.html)

There is Mkdocs ([http://www.mkdocs.org/](http://www.mkdocs.org/)) which is a static site generator targeted towards project documentation. This is more straight-forward and easier to maintain. Both are open source software. You can give them a try and see if you can tailor them to your needs.

</comment><comment><USER27> From what I have seen creating up to date documents is considered superfluous. There are tons of excuses for why people are not updating the documents. Raising a PR for small changes is one of those excuses.

Any tool you find will have it's own version control restrictions. People will use that as a crutch to say - Well, the tool x has steps a, b, c which is too much of hassle for a minor change.

This cannot be solved by using better tools or solutions.

To give examples from my experience:

One of the places where I saw the best documentation was at a bank. Every project came with specific amount of time dedicated to updating documentation. This ensured people don't complain about not having time or the process being tedious etc.

Though as there was no time allocated for peer review of document control - <USER2> did not stop people from being lazy or creating dependencies by knowingly adding generic information in the document.

In worst of places, a software product company - documentation was looked down upon. Everyone was more than happy to "set up time to discuss" than put actual effort in good documentation. As I joined after the aforementioned bank, I had a habit of abhorring meetings and creating up to date document which was shared with the team.

The end result was that people still wanted to have meetings. On being told, I shared document with relevant info with them, they used act surprised and ask to share the document again.

</comment><comment><USER28> Alfresco is an open source Enterprise content management software which has frontend UI called Alfresco Share. The installation is simple using their all inclusive executable which bundles the server, database, and Solr for searching any and all data. It also has this concept of business workflows which you say is a pain point for you. Each document can have workflows assigned to it at which point it has a life of it's own, passing from user to user, emailing itself, tagging itself, etc. There is a learning curve on the dev side, especially if you aren't Java savvy, but once you get the hang of it it's very powerful. I've personally written an API in Lua for syncing documents. I assign bot accounts from the Share UI which are given permission to crawl and sync directories to bot accounts on the server (origin server or external server), and each bot has a job to do on each document it gets, for which it writes and dispatches a service with systemd, then uploads the processed documents back to alfresco. And this is probably 1% of what Alfresco is actually capable of (and my implemention reeks of hackiness, because I hate Java). A team of devs is bound to get more use out of it.

</comment><comment><USER29> TL;DR

* we use Slate for API docs, Google Sites for company documentation (and some Google Docs) * am thinking about using a forum for Q&A e.g. Discourse [2]

Longer:

We've also been going through some thinking on this. At the moment we have an API docs page using Slate, and a Google Sites site which contains a whole raft of documentation. We still have some stuff in Google Docs as well.

Our problem is that not all the team is part of Engineering, so keeping stuff in the repo isn't really ideal (hard to edit). Google Sites is fine but its main selling point is easy access to Google Auth. I have a thread on HN about this [1].

One of the things I have been thinking of is setting up a forum like Discourse [2] for Q&A. Sales often has product-based questions, and vice-versa. In my view, this beats other ways of asking questions (e.g. Slack) because there's no time pressure to answer, and unlike "office-hours" style meetings, it's automatically documented. The only problem is that it is yet another tool.

[1]: [https://news.ycombinator.com/item?id=12540678](https://news.ycombinator.com/item?id=12540678) [2]: [https://discourse.org](https://discourse.org)

<comment><AUTHOR> >The only problem is that it is yet another tool.

That's one of the main horrors we are trying to avoid. We'd like to have a single location to track both kinds of documentation ideally, but realise we are probably going to have to make compromises. Tools which keep things in a standard format will ease migration in the future, so something like Slate would be nice and looks a little better than our current way of "Go view the file in Github".

</comment></comment><comment><USER30> Have done this role a couple of times. You need to make one person responsible for documentation and traceability, because collaborative editing in a wiki like tool without some editorial oversight and organisation leads to problems later on. The tool really doesn't matter, Confluence works well with Jira, pwiki is nice to use, even sharepoint works in a pinch. Heck, I've even used Enterprise Architect and Mindjet MindManager!

</comment><comment><USER31> We have a lot of teams using HelpSite.io [1] for this sort of thing. Originally designed for customer-facing FAQs, we also support the same format for internal knowledge bases. We didn't originally intend to support such use cases, but it turned out a lot of people really like our format for internal company docs as well.

[1] [https://helpsite.io](https://helpsite.io)

</comment><comment><USER32> we have a 2 repository in a gitlab group named "knowledge base", one for public information, one for private information. AND we use a opensource stakcoverflow clone (askbot)

for the git repositories

so that developers works with markdown files, it can be git cloned, it can be grepped, but it can also be rendered in the gitlab website, and most importantly compared to other wiki's or even gitlab's wiki, you can make merge request, so that,

      * if you fear to put an incomplete things, at most it will be in a MR and get reviewed (or never) but at least it's somewhere else than in your head
      * all knowledge merged has been known to be there by someone else, so that it's ot just a wiki full of hidden gems that everyone has buried and nobody else know it exists

This way you have encyclopedic knowledge of "how to put a new project in our CI/CD pipeline" or "list of horrors in project X that you want to delete, but hey that's not so simple and here's why"

the stackoverflow clone on the other hand is here to answer the question everybody ask, from the new employee to the famous "oh why god already do we have to do this command everytime e deploy, I remember John explained it to me one day?" the stackoverflow links extensively to the repository for the "if you want to know more"

we also put as much as possible the "as of march 2017" marker, to always remember people that information rot

things we also do:

every todo in the code has with it a link to the corresponding issue in the tracker, so that when you have 10 minutes to kill, you can check the tracker for "TODO" issues, and grep "TODO #42" in the code

</comment><comment><USER33> We just started using Guru (getguru.com) for our 13-person remote team (trydesignlab.com), and it's been quite effective. Some key features that we love:

\- Simple UI with well-known metaphors like cards and collections — each task or process you have as a company can live in a separate card

\- Tagging combined with lightning-fast search

\- A bookmarklet that allows you to search your Guru knowledge base from any other site you're on (really handy for our support agents who want to look up a manual process without leaving their support workflow in Zendesk)

\- A "verification" workflow that ensures your knowledge base is up-to-date (previously a huge issue in any sort of wiki / document-based solution — you never know if what you're reading is the latest piece of info, or whether it's been updated since). Cards can be assigned to an owner who can be reminded to verify the information on a periodic basis (e.g. every 2 weeks, 4 weeks, etc.), and anyone viewing a card knows whether it's been recently verified

\- The ability to ask questions to individuals / groups within Guru, with answers turned into new cards. Instead of tapping someone's shoulder (or sending them a Slack message), we now ask for info within Guru, organically growing our team's shared knowledge base over time

Overall this has been a tremendous find for us. We had a number of core processes (support, operations, payments, dev, marketing, sales) that lived in various Google Docs (including one monolithic 'Process Guide' doc), but that system was growing extremely cumbersome to use. We strongly considered Confluence, but stayed away after a short trial revealed how bloated the product was — we knew ramping the team up would be a nightmare.

So far Guru's been a winner — would recommend giving it a spin.

</comment><comment><USER34> We use Jingo ([https://github.com/claudioc/jingo](https://github.com/claudioc/jingo)), which is inspired by GitHub's wiki (i.e. is a git repo itself) but with has search function. It's not the best experience on the planet, but it works and is easy to self-host.

</comment><comment><USER35> We have a 'this person better not be hit by a bus or quit on us' policy, or nearly so.

We have people get assigned tasks from other projects from time to time, and they start picking up bits and pieces of the projects from that. And we have an internal wiki that documents some things that people aren't very good about keeping up to date or making it comprehensive.

Also some of our software has gone through rewrites that has cleaned up the architecture and made them easier to understand and work with, without documentation.

But there's still way too much locked in people's heads. We've lost quite a bit of it, too, as people have left over time. I know we're trying to convince a client to shift over to newer versions of our software, because we no longer have anyone that knows anything about the tech the legacy legacy software was written in (it's legacy software that predates our existing legacy software).

</comment><comment><USER36> We are using Jive for all of our internal documentation which works really well for us. For deep documentation, both new document requests and document updates are submitted using a built-in "Ideas" feature. "Ideas" can include plain text, links, images, or videos, and can be submitted by anyone in the company. With this information, we can create a new document, save it in Jive, and then respond to the "Idea" with a "The document is done and here's the link to it" comment. For quick fire Q&A, we've created a page in Jive that uses built-in "Questions" and "Answers" features, again, accessible to anyone in the company. Jive's search feature is pretty robust (keyword-indexes anything you add to it without having to create tags, though you can add tags if you want).

</comment><comment><USER37> For Semantic Synchrony[1], we keep our notes in Semantic Synchrony, a knowledge graph server (written in Java, using Gremlin and Neo4j) and client (Emacs). It allows for selective sharing, and you can connect your private notes to your shared notes however you like -- for instance, I've got a lot of "half-formed hypothesis" notes that I don't think are worth anybody else's time, so I keep those private. It eliminates the dead link problem, and the linearity constraints that trees (let alone flat folders or text files) impose.

It's open-source, with a lot of features, and well documented.

[1] [https://github.com/synchrony/smsn/wiki](https://github.com/synchrony/smsn/wiki)

</comment><comment><USER25> For just files (Word, Excel, CAD, pdf, PowerPoint, etc.) PDM/PLM software is amazing (some top products include Teamcenter, Windchill, & Enovia). They are tailor made to store the versions (including mark-ups) and provide workfows for approval, since they are born out of the need to store CAD parts/assemblies that have strictly defined parent/child relationships, they have no problems doing the same for more simple files.

For the "why" or "knowledge capture", I don't think you can do better than a Wiki and the dominant enterprise Wiki is most certainly MediaWiki. With extensions for improved searching, visual editor, etc., corporate wikis are more powerful and easier than ever to use.

</comment><comment><USER38> Hey, I can recommend [https://zenkit.com](https://zenkit.com) . It´s an all- in-one project management platform with additional handy features for knowledge management. Basically, you can create a collection of all your data and add labels to keep everything organized. The collaborative features allow you to invite your team. Other functions like the global search feature lets you search through the whole app and find information you need (great for FAQ´s - type in a problem and get the documented problem + solution). Also, you can link the tool to over 750 other tools with Zapier and integrate your Google Calendar. Disclaimer: I work at Zenkit.

</comment><comment><USER39> Andy from Tettra here with some shameless self promotion:

My startup is actually working on this exact problem: [https://tettra.co](https://tettra.co)

Our philosophy is that knowledge should be fast to capture, accessible from your communication tools (Slack, etc) and open by default for everyone on your team to suggest edits because most knowledge goes stale in other systems.

Would love feedback from the HN community. We're documentation geeks here and love talking about this stuff.

If you're interested in using the product, we can offer a special HN discount of 20% off the first three months. Just shoot us a message in the chat. I'm on there right now.

<comment><USER17> Your site badly needs a sample of what tettra looks like in action. I have absolutely no clue if you are worth trying out at this point.

<comment><USER39> Thanks for the feedback, and I 100% agree with you. We're working on it.

Want to shoot me an email and I'll send you an update when it's live? andy [at] tettra [dot] co

</comment></comment></comment><comment><USER40> We use a wiki where everybody can update, without any review process.

My personal preference is to try to write the docs from a "customer" perspective, i.e. not for me that I know the system, but from the perspective of another employee that sees an issue and finds my page.

This means:

\- keywords in the title, so the search autocomplete hopefully will find the page in the first results

\- short and sweet, meaning that if a process is more than 3-5 steps, the person won't read/understand it. It's better to improve the process than to provide better docs

\- when I read something that doesn't work, I do my best to fix it, even if I'm in a rush

</comment><comment><USER41> I work for a Fortune 50 and we use Confluence. For us it works well, because we have more teams here than I can count, and people move between teams (and teams move between business units) all the time. So the standard rule is that if you build something, you are expected to document it in the Wiki.

About the only downside is that sometimes documentation gets outdated and you have to hunt down the current version of a process, but that beats the alternative, where nothing is documented.

</comment><comment><USER42> You can try forcing people to write documentation for every new feature they make. Do not let merging without a documentation. Added major feature - <USER3> them to write a post about it on some documentation platform (for example, Confluence). Old methods could be documented whenever you interact with it. Step by step, slowly but surely you're gonna cover your project with good doc

</comment><comment><USER43> Small IT team, in public health, using MediaWiki on Windows.

Search was underwhelming for our 400 or so pages, so we tuned an old MW extension called RigorousSearch, see [https://github.com/thomasswilliams/RigorousSearch- MediaWiki-...](https://github.com/thomasswilliams/RigorousSearch-MediaWiki- extension-updated-for-MediaWiki-1.28)

</comment><comment><USER44> We used MediaWiki for documentations, however we were very unhappy with it. We have almost all an computer science background, but did not like the MediaWiki syntax and the interface.

We created Teamemo ([https://teamemo.com](https://teamemo.com)) because knowledge management should be easy, fast and fun.

<comment><USER25> Did you try MediaWiki's Visual Editor?

</comment></comment><comment><USER45> We use a mixture of Asana and Google Drive.

Asana is an amazing tool to store information as well as create a unified workspace where all members can share work, collaborate on projects/tasks, or have a knowledge repository. Would definitely recommend!

</comment><comment><USER46> Hey everyone :)

My name is Omer and I am currently building a knowledge documentation solution called Scribe (formerly Mindflow.ai).

Scribe runs in the background of your development environment and creates documentation based on your workflow. Additionally, we allow for users to easily create their own documentation in an intuitive, easily searchable fashion.

During Y Combinators Startup School we launched our Alpha product, and we are currently building the beta version of our product. If this sounds like it can help, you're more than welcome to sign up!

[http://mindflowai.com](http://mindflowai.com)

</comment><comment><USER47> Tangent: I remember the first time I used Quora, thinking that it had a great model of content discovery and collaborative answering — that it would be fantastic if they let companies deploy internal versions of it. Now that Quora has been around for 7 years and has clearly decided not to do this but instead to make money by , does anyone have any guesses as to why they decided not to?

<comment><AUTHOR> An alternative for this case for you might be the enterprise version of StackOverflow? We have that internally at the company, but it's not particularly friendly for the use cases our small team has.

</comment></comment><comment><USER48> Nuclino has been a winner for us.

</comment><comment><USER49> OneNote - it might not be excellent, but it's got a super low barrier to entry.

</comment><comment><USER50> Documentation, the thing most agree is a good idea, but nobody wants to do. It's a nuanced subject.

TOOLS

Until recently, we used a combination of Documentum and EPM (Enterprise Project Manager). The intended workflow was to have all documents start in EPM as part of the initial project implementing/building whatever product. When the project closed then the developer was to copy the documents across to Documentum. It was simple in concept. Due to technical shortcomings, it was not. I stopped even trying around 3-4 years ago.

Documentum was a technical horror show. Ten years ago, and for a long time afterward, it supported only one browser, I believe IE. Architecturally, it seemed to use a Java applet for login and, upon successful login, it downloaded an app. Of course, the app wanted a specific version of the JRE locally. But, the real nightmare was authorization. One day you had permissions to update "cabinets" for your documentation. The next day you didn't. No explanation.

EPM had authorization issues similar to Documentum's with the added bonus of a project's link not working all the time. One day you could use the link, which the PM sent out at the start of a project, to access the project workspace. The next day you couldn't find the project workspace.

Currently, we're moving into a SharePoint site that is more promising. It provides Wiki capability for quick reference items, as well as, a repo for more involved documentation. Apparently, it provides versioning, but we haven't explored it yet.

TYPES

Part of the problem with the subject of documentation is that different groups want different types of documentation. Somehow, the developer is supposed to understand the needs of each group and produce a document tailored to those needs.

Operational people want information on jobs and job dependencies. Support people want lists of things to check and try in case errors occur. (Developers are inherently bad at documentation for customer support. The problem is that we know how the system is supposed to work, so we're blind to its shortcomings until a user finds them. Rather than my writing a support document upfront, I prefer meeting support people periodically so they can ask questions and develop their own document.)

Management wants everything, all at once. They want it in the amount they want it, summarized, but detailed enough that they can feel like they're a help during outages and architectural meetings alike.

Users want to know every step for every task they need to accomplish, as well as, what to do in any anomaly, like not paying attention to dialog boxes and hitting "Save" to commit information that they didn't want to commit.

ISSUES

The major issues, for me, are keeping documentation current and writing for such varied audiences, nevermind having the storage system working against me.

</comment><comment><USER51> Anybody tried Dropbox Paper?

</comment><comment><USER52> I work with a large (multi-billion, multi-national) tech company that has a big problem with knowledge management.

My job involves consulting to lots of customers in different industry sectors, and the one small, sad comfort is that almost no one does documentation management well.

Currently if I want some specific tidbit of information I can consult: our knowledge base (bespoke), our collection of product PDF's, salesforce documentation repository, quickbase documentation repository, an aging wiki (twiki), private stash of documents created by colleagues, mailing lists (a subset of which are indexed and accessible from the wiki, otherwise you can only go back as long as you've been with the organisation, and hope you've been subscribed to all the right lists), sharepoint (meant to replace the wiki, but two years later, and no), our learning management system, yammer (seriously), bugzilla, our customer support forums ... or I think about who might know the answer and email them.

My advice as a result of this:

Even if you don't hire a librarian, at the very least appoint a curator -- ultimately you need someone who owns your documentation, and the design and management processes around it, and can encourage/force people to update it, continually reduce redundancy, improve quality, etc etc.

Be wary of your current thinking " We'd like to have a single location to track both kinds of documentation ..." \- specifically that there are only two kinds of documentation. Or indeed only n kinds of documentation.

It's easy to fall into the 'let's build a new system to encompass the others' trap (insert obxkcd) and hard to fight it. If you _do_ go down that path, ensure the content from old systems are fully migrated (3 years after it's been 'retired' our twiki is still the best place to go for some things). This is hard - it's rarely budgeted, and involves tedious and/or impossible work.

I suspect it's impossible to have a single system that will manage all your institutional knowledge -- I know many people are working on solving this from both directions, but I don't believe there's anything suitable now.

Whatever you choose it needs to have a low cost of entry for non-technical staff to update, while not frustrating your technical types. This is hard to find.

In 2007 I set up DekiWiki (based on mediawiki, but since abandoned) for a medium-sized gov agency -- in 2013 when I caught up with that team I was surprised they were still running it. It was internal only, so they weren't so worried about lack of patches, and they hadn't found anything better in the interim.

Out of _that_ experience I'd suggest that whatever you do go with, make sure it's easy to extract your data, and that you maintain your own full copies of product documentation just in case the product is abandoned. Also - don't assume products won't be abandoned. MindTouch / Dekiwiki [1] was _big_ in the mid 2000's.

[1] [https://en.wikipedia.org/wiki/MindTouch](https://en.wikipedia.org/wiki/MindTouch)</comment>


Boeing expresses regret over ex-pilot's messages on 737 MAX software - <AUTHOR> https://www.reuters.com/article/us-ethiopia-airline-boeing/boeing-expresses-regret-over-ex-pilots-messages-on-737-max-software-idUSKBN1WZ0MA <comment><AUTHOR> The spin in Boeing's response to this left me dizzy.</comment>


From the Agony of JUnit to the Ecstasy of RSpec - <AUTHOR> http://asymmetrical-view.com/2010/02/19/maven-rspec.html

<comment><USER2> It's probably good to be able to standardize on something to test both Java and Clojure code. I have to wonder how much Clojure you can cover from JRuby, though, since you have to go to extra lengths to make all of your Clojure fns exposed in generated classes. The clojure.test package is pretty good in my experience (up there with Rspec, but it feels lighter).</comment>


How someone spent "pennies" to troll women, people of color via promoted tweets - <AUTHOR> http://arstechnica.com/business/2015/05/05/how-someone-spent-pennies-to-troll-women-people-of-color-via-promoted-tweets/ <comment><USER2> So they create a new Twitter account, wait for a discount code for Twitter ads, spend pennies on the dollar for an offensive tweet to promote it and end up trolling the users.

It didn't use any racist words, so it passed a word filter. It just said that white people are being discriminated against and had to protect themselves from discrimination and violence. Change the white word to black and it could have been from Ferguson or Baltimore protesters. I've seen a lot worse on Twitter than this troll wrote. I think he made his point that the system can be exploited to troll people in certain groups. I hope Twitter makes some adjustments and lets a human being read the promoted tweet before it is promoted.</comment>