Safe and Secure Drivers in High-Level Languages [video] - <AUTHOR> https://media.ccc.de/v/35c3-9670-safe_and_secure_drivers_in_high-level_languages <comment><USER2> Personally, I'd call this writing "Safer and More Secure Drivers in High-Level Languages", because there are still unsafe operations going on (for DMA, etc.): [https://github.com/ixy- languages/ixy.swift/search?q=Unsafe](https://github.com/ixy- languages/ixy.swift/search?q=Unsafe)

While it's great that you get improved safety (and often nicer and easier to reason about code) by using something other than C, you can still have memory safety issues from cavalier or incorrect usage of unsafe APIs, since they undermine the guarantees the language provides with regards to correctness.

Also, unrelated: does anyone actually have the slides (you know, the presentation file with text in it, rather than the mp4 that CCC is offering me) for this presentation? It's really annoying to scrub through a video to find stuff on a slow internet connection :(

<comment><USER3> I've added them to the git repo: [https://github.com/ixy-languages/ixy- languages/blob/master/s...](https://github.com/ixy-languages/ixy- languages/blob/master/slides/35C3.pdf)

<comment><USER4> Direct link for download (GitHub won't render the entire PDF by default): [https://github.com/ixy-languages/ixy- languages/raw/master/sl...](https://github.com/ixy-languages/ixy- languages/raw/master/slides/35C3.pdf)

</comment></comment></comment><comment><USER5> Another option is to use a memory safe subset of C++ [1]. It should be less work to migrate existing C drivers as (reasonable) C code maps directly into the safe C++ subset. And the migration can be done incrementally with corresponding incremental safety benefits.

[1] shameless plug: [https://github.com/duneroadrunner/SaferCPlusPlus](https://github.com/duneroadrunner/SaferCPlusPlus)

<comment><USER6> Only if the team plays along regarding static analysers and compiler warnings as errors.

<comment><USER7> Wouldn't you simply enforce this with automation if you were making a serious effort? It's already quite common for github PR's to require myriad CI tests to pass before anything can be merged... those can incorporate static analysis and warnings as errors.

<comment><USER6> Still needs to have the buy-in from the team.

Try to be that clever guy putting such gates into place without having the team be on the same wave length.

Github is a bubble, there are tons of software projects out there, using a myriad of build infrastructures, or even just doing plain old IDE builds (yes I know, but it is as it is).

<comment><USER7> Convincing your team to switch languages is infinitely more difficult than adding infrastructure to enforce good hygiene. So I don't really see what your point is, it's moot.

<comment><USER6> This whole thread started about enforcing behaviors that are largely ignored by enterprise developers outside HN bubble.

In no point there was a mention of switching languages.

<comment><USER7> Ah, this had left an lingering impression in my mind:

"Why do that, if there are languages that do it by design?"

But I see it wasn't your comment, my bad :)

</comment></comment></comment></comment></comment></comment></comment><comment><USER3> Code on GitHub: [https://github.com/ixy-languages/ixy- languages/](https://github.com/ixy-languages/ixy-languages/)

Discussion about the C version on GitHub in 2017: [https://news.ycombinator.com/item?id=16014307](https://news.ycombinator.com/item?id=16014307)

<comment><USER8> Great work! Do you have by any chance the C driver latency measurements? It would be nice to have them on the same graph as those for Rust and other languages.

<comment><USER3> Would look the same as rust; I haven't run the full measurement with all the packets. But sampling 1k packets/second yields the same result for C and Rust.

You can't really get a faster speed than Rust here; the only minor thing that could be improved is the worst-case latency by isolating the core that the driver is running on (isolcpus kernel option) to prevent other random interrupts or scheduling weirdness. But that optimization is the same for all languages and should get rid of the (tiny) long tail.

<comment><USER8> Thanks!

</comment></comment></comment></comment><comment><USER9> Great direction! One of the lesser known but really nifty bits of NeXTStep was DriverKit:

[http://www.nextcomputers.org/NeXTfiles/Software/OPENSTEP/Dev...](http://www.nextcomputers.org/NeXTfiles/Software/OPENSTEP/Developer/DriverKit/DriverKit.pdf)

Objective-C was a great match for driver development, devices tended to have a very natural OO flavour to them and naturally sorted into classes.

Putting things in user-space seemed a natural extension, that sadly didn't happen at the time even though Mach had the hooks for it (we never got user- level pagers either, which would have rocked together with garbage collected languages). There certainly didn't seem to be a good reason why I had to reboot the machine and wait for fsck when there was a minor bug in the little driver I was writing to talk to an EISA printer controller that had nothing to do with the basic functioning of the system...

(Why would a printer controller be on an EISA controller, you ask? It directly drove a Canon Laser Copier, so, yes!)

Oh, and not surprised by the abysmal Swift performance. Alas, Apple's marketing of Swift as a "high-performance" language has been very successful despite all the crushing evidence to the contrary.

<comment><USER2> > Objective-C was a great match for driver development, devices tended to have > a very natural OO flavour to them and naturally sorted into classes.

How do you feel about the current state of driver development on macOS, with Objective-C basically being replaced with Embedded C++ with partial reflection?

<comment><USER9> It was part appeasement of the "never Objective-C" crowd (hello CoreFoundation, hello CocoaJava, hello Swift) and part the exact sentiment discussed here, that you cannot possibly do kernel development in a higher level language.

What I heard (quite some time ago) is that this move is now seen as a mistake.

<comment><USER10> ObjC was definitely seen as a dead-end at the time: it would either be replaced with Java, or maybe Mac developers would just stick with Carbon and C/C++. Either way, all driver development (on classic Mac, Windows, Unix) was in C, and C++ would be much more familiar than the "weird obsolete square- brackets NeXT language"

I made a post a few years ago discussing the issue:

[https://news.ycombinator.com/item?id=10006411](https://news.ycombinator.com/item?id=10006411)

</comment></comment></comment></comment><comment><USER6> As addition to the talk,

Android Things userspace drivers are done in Java, and since Treble, it is also possible to write Android drivers in Java.

MicroEJ and Windows IoT Core also allow for such capabilities.

Loved the talk.

</comment><comment><USER11> It is disappointing that Ada seems to be completely shunned from all discussions of safe high-level languages despite having both a track record (several successful safety critical systems) and a unique feature set (multiple compilers, provably correct subset, ranged subtypes).

Is it purely a matter of what is well-branded?

<comment><USER12> Ada also suffered from bad roots and bad timing. We're off a decade of simple dynamic languages. Ada seems like an old and immense ruin. Maybe there will be a julia/rust equivalent for Ada.

<comment><USER11> And it is very true -- Ada never truly gained any foothold in anything but situations where it truly delivered upon a requirement.

If anything, I can take solace in the fact that it is very much alive despite the exaggerated rumors of its death. I'd encourage everyone to try it out and steal all the ideas.

</comment></comment></comment><comment><USER13> > Our drivers in Rust, C#, go, and Swift are completely finished, tuned for > performance, evaluated, and benchmarked. And all of them except for Swift > are about 80-90% as fast as our user space C driver and 6-10 times faster > than the kernel C driver.

Interesting. What is the reason for higher performance of user space C driver (and the other user space drivers for that matter) when compared to the kernel C driver? Will this hold for all driver types or is this a rather uncommon property of this particular kind of driver?

<comment><USER14> > A main driver of performance for network drivers is sending/receiving > packets in batches from/to the NIC. Ixy can already achieve a high > performance with relatively low batch sizes of 32-64 because it is a full > user space driver. Other user space packet processing frameworks like netmap > that rely on a kernel driver need larger batch sizes of 512 and above to > amortize the larger overhead of communicating with the driver in the kernel.

<comment><USER9> Interesting!

As our devices have gotten faster, the user-space/kernel boundary is becoming more and more of an issue. I was shocked when my supposedly super-fast MacBook Pro SSD (2+GB/s) was only giving me around 250MB/s.

Turned out mkfile(8), which I was using without thinking much about it, is only using 512 byte buffers...

[https://blog.metaobject.com/2017/02/mkfile8-is-severely- sysc...](https://blog.metaobject.com/2017/02/mkfile8-is-severely-syscall- limited-on.html)

<comment><USER15> The main thing affecting performance at the userspace/kernel boundary these days is Spectre and Meltdown mitigations, FWIW...

<comment><USER16> No, userspace/kernel transitions are always and will always be slow. Every time it happens you've got to do a context switch which is super-expensive + cache unfriendly. You also pay a penalty keeping the kernel mapped at all times in terms of more pressure on the TLB but due to Spectre and Meltdown mitigations the kernel has actually been unmapped hurting the performance of switching into kernel further although this will be undone eventually.

</comment></comment></comment></comment></comment><comment><USER3> Previous discussions about our Rust and Go drivers:

[https://news.ycombinator.com/item?id=18405515](https://news.ycombinator.com/item?id=18405515)

[https://news.ycombinator.com/item?id=18399389](https://news.ycombinator.com/item?id=18399389)

</comment><comment><USER17> What would make device drivers safer:

\- Microkernel OSes - one driver failing is okay, because it runs in mostly in user-space. The big gotcha in microkernel design is transactions that touch multiple components. Some sort of across-driver transaction API is needed (start, commit, rollback) in order to undo changes across several userspace subsystems.

\- A standard language (like the talk suggests) and shipped as portable bytecode to run on a VM or compile to native, so that drivers are portable and runnable without knowing the architecture.

\- Devices themselves containing OS-signed drivers rather than each OS having a kitchen-sink installation of all drivers. Each bus would have an interrogation call to fetch the driver.

</comment><comment><USER18> Is there a plan for letting applications using the normal networking APIs see this, or is it currently just for "process raw ethernet packets in userspace" kind of apps?

(The latter is a great thing to have built, of course, just thinking aloud about how this might replace existing drivers)

<comment><USER3> Yeah, I'd love to port a Go TCP stack like [https://github.com/google/netstack](https://github.com/google/netstack) to use our driver to build a microkernel-style service offering network connectivity. Running taps ([https://datatracker.ietf.org/wg/taps/about/](https://datatracker.ietf.org/wg/taps/about/)) on top of that would be ideal for a modern setup. But that's a lot of work...

</comment></comment><comment><USER6> Swift loosing heavily due to reference counting GC instead of a tracing GC.

Now I have a talk to point out to those that always trump that referencing counting GC are so much better than tracing ones.

<comment><USER19> Their notion of "better" may differ from yours. Do you actually know people who claim that reference counting is _faster_ than a marking GC?

Reference counting can be better in terms of ease of implementation, cross- language interoperability, and by reclaiming memory immediately when the last reference to it disappears.

<comment><USER6> Reference counting is only better in terms of ease of implementation.

Hence why it is usually one of the earliest chapters in any CS book about GC algorithms.

Reclaiming memory immediatly only works in simple data structures. Naive reference counting impletations have similar stop-the-world effects when releasing relatively big data structures, which can even lead to stack overflows, if the destructor calls happen to be nested.

In case of Objective-C and Swift, Objective-C tracing GC project failed due to the underlying C semantics, thus they took the 2nd best approach of enforcing Cocoa retain/release patterns via the compiler, which only applies to a small set of Objective-C data types.

Swift naturally had to support the same memory management model, as means to keep compatibility with the Objective-C runtime.

<comment><USER16> Eh. In practice reference counting solves 90% of the problem and keeps memory usage low at all times. It's part of why Java programs are so hard to tune for performance & end up eating so much RAM. If you don't believe, compare Android & iOS where even though Android has enormous financial & competitive pressure to improve the performance they still end up requiring 2x the amount of RAM that iOS does which is partially driven by the choice of Java.

<comment><USER6> People keep referring to Java to talk bad about tracing GCs.

The fact is that Java isn't the only game in town, and all GC enabled system programming languages do offer multiple ways to manage memory.

Value types, traced GC memory references, global memory allocation, stack values, RAII, untraced memory references in unsafe code blocks.

Not every OOP language is Java, nor every GC is Java's GC.

Additionally, not every Java GC is like OpenJDK GC's, there are plenty to chose from, including soft real time ones for embedded deployments.

As for Android, it is a fork still catching up with what Java toolchains like PTC/Aonix are capable of, all because Google decided it could do better while screwing Sun in the process.

<comment><USER15> > and all GC enabled system programming languages do offer multiple ways to > manage memory.

Since "GC-enabled system programming languages" is an oxymoron, a claim about what such languages may or may not include is just not very useful. But it's definitely the case that properly combining, e.g. "traced GC memory references" and RAII including deterministic deallocation for resources is still a matter of ongoing research, e.g. [https://arxiv.org/abs/1803.02796](https://arxiv.org/abs/1803.02796) That may or may not pan out in the future, as may other things such as _pluggable_ , lightweight GC for a subset of memory objects, etc., but let's stop putting lipstick on the pig that is obligate tracing GC.

<comment><USER6> An oxymoron only in the minds of anti-tracing GC hate crowd.

\- Mesa/Cedar at Xerox PARC

\- Algol 68 at UK Navy computing center

\- Modula-2+ at Olivetti DEC

\- Modula-3 at Olivetti DEC/Compaq/HP and University of Washington

\- Oberon, Oberon-2, Active Oberon, Oberon-07 at ETHZ

\- Oberon-07 at Astrobe

\- Component Pascal at Oberon microsystems AG

\- Sing#, Dafny and System C# (M#) at Microsoft Research

\- Java when running AOT compiled on bare metal embedded systems like PTC Perc and Aicas Jamaica

\- D by Digital Mars

\- Go at Google (Fuchsia) and MIT (Biscuit)

Lets stop pretending reference counting is the best of all GC algorithms, in spite the fact that is quite basic and does not scale in modern multi-core NUMA architectures.

<comment><USER16> No one is saying that reference counting is the best. What I am saying is that reference counting tends to offer a good set of advantages (predictable memory performance, no hogging of memory, no pauses) for a minimal cost (more frequent GC, more overhead to store reference counts).

The comment about "does not scale in multi-core NUMA" only applies if you have objects that are shared between threads because otherwise there's no atomics going on. For example, Rust has a generic ref-count mechanism that automatically uses atomic operations for ref-counts when an object might be shared between threads but otherwise does simple arithmetic. Non-atomic refcounts are most likely also going to be faster than any other global GC algorithm. Other languages require explicit differences but are still able to offer the same thing.

The fact of the matter is that the majority of objects do not require expensive GC of any kind & can live on the stack or have explicit ownership guarantees. Choosing defaults where everything might be shared is not a good default for systems languages as it pessimizes memory usage & CPU performance to a drastic degree.

That being said, GC does have its place in all manner of applications and has other advantages like making developers more productive which isn't a bad thing but these are domain-specific decisions. There are plenty of techniques - reference counting, memory pools, static memory allocation, various GC algorithms, etc, etc. Each has tradeoffs & every single GC system I've encountered means variable latency/stop-the-world and greedy memory usage (optimized for the 1 application). That's valid in some domains but certainly isn't desirable. If there were an awesome GC system like you claim that could perform that well it would have been deployed already to inemurable applications like all Java vendors, Javascript VMs, C#, etc, etc. It's an extremely complex problem.

Most of your links are niche commercial systems or even pure academic research systems. They're not proof of anything other than GC being possible to implement for various languages/machines which isn't a claim that's been disputed at all.

> Go at Google (Fuchsia)

AFAIK Fuchsia does not use Go for any systems-level portions. Those are written in C/C++/Rust last time I checked (with Rust being the official default going forward). Do you have any links to the contrary?</comment></comment></comment></comment></comment></comment></comment></comment>


Venezuela's cryptocurrency: Petro Whitepaper [pdf] - <AUTHOR> http://www.elpetro.gob.ve/Whitepaper_Petro_en.pdf <comment><USER2> What a horrifying idea to back a cryptocoin with oil. Oil is really not a long-term value store as technology improves and the world goes electric. It may be viable for 20-100 years, but tying a coin to a resource -- especially naming a coin after a resource -- will only lead to difficulties and confusion down the road. Would you ever fill up your tank with BTC? Would you ever buy a coffee with Tether (scam) ? Presuming you answered definitively "no" to these two questions, then why would you ever use a cryptocoin to buy petroleum or accept a coin in exchange for petroleum. Maybe the idea is lost on me, but tying a globally and internationally exposed digital asset that feels the variability of markets and wake/sleep cycles of humanity to something as dirty and short-term as oil seems to just be asking for problems.

It's a tempting idea because Venezuela exports a lot of oil, but you may as well try and tie a coin to oxygen and see how you do.

<comment><USER3> Most dominant currencies of the past centuries were tied to real commodities with intrinsic value -- eg, gold, silver for instance. There is nothing horrifying about a commodity-backed currency, especially when compared to the "alternative" \-- the international reserve currency, the US dollar, is backed by "confidence" and is created out of thin air -- that we are now stuck with.

I haven't read Venezuela's crypto offering in detail, but Venezuela's problem, as I see it, is most investors/traders won't trust socialist Venezuela to honor their contracts when push comes to shove.

<comment><USER2> Commodity-backed coins are great, but why oil of all things? It's dirty, low- tech, actively polluting the air, and running out. It looks like someone just looked at a list of chief exports and thought "yeah I can make a new currency for that"</comment></comment></comment>


SQL Injection on eBay.com.au subdomain - <AUTHOR> http://blog.internot.info/2014/05/sql-injection-on-ebaycomau-subdomain.html

<comment><USER2> Does Australian Ebay have a different security team and different management?

5 days to fix SQL injection seems quite slow.

SQL injection at all is quite embarrassing. Why was it not caught and fixed by routine QA or auditing?

<comment><USER3> Large organizations sometimes manage to institutionally forget that they ever had some assets:

[http://www.theregister.co.uk/2001/04/12/missing_novell_serve...](http://www.theregister.co.uk/2001/04/12/missing_novell_server_discovered_after/)

</comment></comment><comment><USER4> Oh noes, they're going to steal all of the Kangaroos!!

<comment><USER5> Hang on, you're sounding _flippant_ about this. Stealing all the kangaroos would be a very serious thing, you know.

<comment><USER4> Yes I realize that. How would Australians be transported?</comment></comment></comment>


What Does a Software Development Manager Do Next? - <AUTHOR> I've been a software development manager for more than 2 years now. I used to code more at the beginning then I let me team focus on code, technical aspects as I trust them and mainly I would like them to have the freedom.
I sometimes write API code but very basic ones.
Most of my day is spent on meetings, support to end users, writing user stories, and some admin tasks like looking for interns, interviewing, sorting out licenses, making sure servers are up and get infrastructure team to look into it, etc etc,<p>so fast forward 2 yrs I have lost technical sharpness, so I cant be a programmer anymore, I'm not a a BA neither a PM. I champion agile practices in my team but I'm not an agile coach though. At this stage I feel a bit lost. I feel like I'm not good at anything. where do I go from here? <comment><USER2> Where do you want to go? It sounds like you are implying you want to get back to the technical work? If the job does not allow, then you may need to find a role. Or is the concern that you are losing touch with the _current_ tech and can't make technical decisions like you used to? Or are you asking what is the next step as a technical manager?

If you feel that you aren't good at anything (which clearly is not true given that you've fostered your team) then what is it that you want to get _good_ at?

<comment><AUTHOR> The team is mostly settled, we have database programmer who pretty much handles everything, programmers who work directly under business analysts taking tasks from them. I find myself not adding any value to the team. In other words, apart from day to day admin tasks and some occasional instances I am hardly needed. So I'm thinking what next? Not looking to be a full time developer,my health condition doesn't allow for it anymore.

<comment><USER2> > I find myself not adding any value to the team.

If the team is as _optimized_ as it can be, then it may be time to move on from this team to another where you can add value. This sounds like an obvious statement, but without knowing much about your organization this may or may not be a viable option.

But again to my original, what do you want to do? How do you want to grow? What impact do you want to have? If you have the organization structure such that you can make a case to join a new team and help them grow and/or help with strategy for the organization as a whole (which will touch multiple teams), then there is that path as well.

If you want to transition organizations altogether then the above still holds.. in the sense that you need to define your own goals and then find an organization where you can develop them and have an impact.

</comment></comment></comment><comment><USER3> >so fast forward 2 yrs I have lost technical sharpness, so I cant be a programmer anymore

Do you really have such little faith in your ability to relearn things?

<comment><AUTHOR> I more of lack of desire to be a full time programmer, mainly due to my health issues.

<comment><USER3> I see. My apologies.</comment></comment></comment>


Redash – Connect to any data source, easily visualize and share your data - <AUTHOR> https://github.com/getredash/redash <comment><USER2> Hi! I'm the author of Redash :) Nice to see Redash at the top of HN.

I'll be happy to answer any questions about Redash, open source and about my journey to make Redash a self sustainable project.

For any questions that don't fit on HN, feel free to reach me at arik at redash.io.

<comment><USER3> We LOVE REDASH at our company. Lack of adequate developers (the neverending startup saga) meant not everyone got what they wanted, fast enough. Redash has changed that for us. We use it extensively across all our teams (operations, sales, marketing etc). Thanks!

</comment></comment><comment><USER4> I tried Redash some time ago while I was looking for a very simple "BI" like tool that our non-technical or low-technical colleagues could use (people in Client Satisfaction team).

I settled for Metabase ([http://www.metabase.com/](http://www.metabase.com/) , [https://github.com/metabase/metabase](https://github.com/metabase/metabase) ), mainly for the ease of use and installation (a simple java -jar metabase.jar does the trick).

Redash was a close second (one advantage at that time was the ability to have users that could only read queries), but after fighting with it for a while in order to install it, I tested Metabase and haven't looked back since.

<comment><USER2> (Redash author)

Having a single executable is definitely the best, and I wish we had one for Readsh... trying to compensate on this with the Docker and cloud (AWS / GCE) images for easy setup.

<comment><USER5> How about bitnami? They provide Redash virtual machines and installers already?

<comment><USER6> Or a cloudron app.

</comment></comment></comment></comment><comment><USER7> Looks great! What about open-sourced Superset[1] from AirBnB? Both looks great, but Superset seems to have a superset of Redash features (bad pun intended).

[1]: [https://github.com/airbnb/superset](https://github.com/airbnb/superset)

<comment><USER8> Metabase is another great open-source tool: [http://www.metabase.com/](http://www.metabase.com/)

<comment><USER9> Metabase is my new favourite tool: I've probably use it every day to answer some question instead of dropping into the Rails console. Things like: what percentage of orders come from Stripe vs Paypal? Or export a list of users that match this criteria. Or display the top 10 users by some metric.

It makes creating BI charts from a database easy enough that a non-developer can do it. But when the going gets tough, I can still drop down to hand-coded SQL. It was also an easy installation on Heroku.

It's not entirely perfect (combining multiple series on one chart is harder than it needs to be; can only pull data from databases not APIs) but it's been a success so far

<comment><USER10> Hey I work on Metabase, thanks for the kind words and feedback.

Regarding pulling data from APIs, there's nothing inherently stopping us (or you) from pulling data from APIs (in fact that's what the Google Analytics driver does). Are you looking for integrations with specific 3rd party APIs, or an easier way to integrate with your own APIs?

Feel free to respond here, email me, or file a GitHub issue. Thanks.

</comment></comment></comment></comment><comment><USER11> Redash is fantastic. Things like scheduled queries just work, and the AMI is a great way to get up and running quickly. The docs are decent and the upgrade script works well. I also really appreciated the (optional) Login with Google feature and the ability to limit it to certain email domains (we use Google Apps, so it worked really well). We've been trialling it casually in our engineering and data science teams.

However, the latest version is a little rough around the edges. Bugs like this one [https://github.com/getredash/redash/issues/1520](https://github.com/getredash/redash/issues/1520) (can't delete users, even at the CLI) and this one [https://trello.com/c/6siqvcxh/39-admin-should-be-able-to- dis...](https://trello.com/c/6siqvcxh/39-admin-should-be-able-to-disable- delete-a-user) (there's no option to delete a user in the admin UI, requested since September 2015) make fall just short of production-ready.

<comment><USER2> (Redash author here)

I was really hoping to address this in the upcoming v1.0.0 release, but eventually didn't want to delay it as it was in the oven for too long anyway. It will probably be addressed in the release after.

Beside the ability to delete a user have you experienced other rough edges in the recent releases? We strive to improve the stability and reduce the rough edges with every release, so really want to hear about your experience.

<comment><USER11> Hey thanks for replying.

We noticed a couple of UI irregularities (for example when editing charts the contents or axis labels would disappear). Unfortunately I don't have anything more specific for you right now. I did notice when deleting users it seems that they (and their dependent db objects) actually deleted from the database (rather than flagged as deleted) - for traceability it would be great if they weren't removed altogether.

Though to balance things, let me tell you what we loved about Redash. The user/group/source model is really nice - I appreciated being able to give only our admin group users access to a data source configured with our database master user credentials. The Google OAUTH functionality (and really clear documentation around this) and pre-rolled AMI were a huge plus, allowing me to get our team up and running really quickly. The same goes for the Lets Encrypt instructions (we all use certbot regularly but no-one had to dig through Redash's config files blind - the docs were spot on). The ability to fork queries - superb. Data export was also fantastic.

</comment></comment></comment><comment><USER12> We use Redash via ([https://redash.io/](https://redash.io/)) for a truly hideous amount of stuff at our startup. We use it for almost every operations dashboard, and we've also prototyped out a task allocation and field agent management tool using them and Zapier.

They're really great, I'd highly recommend them to anyone.

<comment><USER2> Thanks <3\. Always enjoy seeing what Eli does with Redash & Zapier. Really looking forward for when the Zapier integration becomes official and goes into their directory.

</comment></comment><comment><USER13> We've been using it on a project to aggregate clinical trial data from many different sources ([https://opentrials.net](https://opentrials.net)) and it has been great!

It allows researchers (not necessarily devs, usually medical doctors) to peek our raw data, and it's a great excuse for them to learn at least the basics of SQL. The response has been great.

We also use it to do some small data checks on the data quality, with alerts sent to our Slack.

Highly recommended.

<comment><USER2> Thank you for proving it's not only for "tech people", and that "normals" can pass the SQL barrier too :) (and learn a useful skill on the way)

</comment></comment><comment><USER14> We have been using Redash at our company for almost a year now. Every single release just proves how promising the project is. You can make useful dashboards in minutes. Support for multiple databases is amazing. We are using it with multiple PostgreSQLs, Redshift, MongoDB and InfluxDB.

The most valuable feature is alerts though. I work at an ecommerce and operations heavy company where we have tons of connected components. Where alerts come really handy for us is that anyone in the organization can add quick alerts for proactive monitoring of events recorded in on-field ops and act when things go bad. This almost like building a feature on one of the internal tools, just doing that yourself without any engineering support. This comes in really handy.

Kudos to the team! Looking forward to some more amazing stuff in Redash!

</comment><comment><USER15> I'm a marketing consultant and Redash has worked its way into my preferred stack for analytics: Segment + Redshift + Redash.

It replaces event- and user-tracking tools like Mixpanel, Heap, Woopra, and others. Those are just UI layers on top of SQL queries. If you or your marketers know (or can learn) even basic SQL then Redash is what you want.

</comment><comment><USER16> I really like Redash, it's one of the early tools that introduce this concept of turn SQL into chart to developers, and also teach developers to learn and write better SQL, altogether without any cost. I evaluated Redash during my past company back in 2013 (we were also using Tableau), but due to some Redash's lack of features (no support for filters, lack of permission control, sporadic performance), we went and build something inhouse with similar approach (turn SQL into charts).

And inspired by the same path Redash founder took, that internal project turned into a startup by itself.

We're relatively new but getting good momentum. Some of our customers went with us after evaluating both. While we don't have a self-hosted open-source version, our pricing only starts at 49$/mo for up to 5 users (pretty affordable for startups IMHO).

You can check it out here: [https://www.holistics.io](https://www.holistics.io)

<comment><USER17> Hello! Redash is great if you are looking to host it in-house and have the engineering resources to set-up and maintain it.

If you are looking for an affordable SaaS alternative, you may want to look into Holistics (www.holistics.io). (Note I'm the other co-founder here with huy)

Besides supporting native SQL, Holistics is designed to address the gaps of SQL for common business reporting use-cases (flexible way of passing user inputs as parameters into report query; supporting if-else capabilities in SQL, reusable query templates and records/columns based access control for users/user groups).

This makes it easier to manage and reduce the management of duplicate SQL query syntax across multiple reports/dashboards (especially multiple UNIONS/Case-Ifs statements). We also have our own DSL for you to configure in more details how certain charts should look like (beyond the normal coloring).

A common problem we also see is that most data-related work is not just visualization, mainly for the reason that most data are not structured/formatted in the right table structure as most companies start off without a data warehouse.

While most query tools requires customers to work with a separate ETL/warehousing tool, we've built an integrated approach towards data reporting and data preparation. Insights from our data reporting module (reports with expensive joins, long query times, non-optimal table structures) provides your data analyst inputs to easily move, map records, and transform data without technical engineering knowledge. Data in Google Sheets or CSVs with their data sync automatically (incrementally or full) to their database.

And all this is done with the data not leaving our customers' database (we don't warehouse their data. Our data reporting module works directly with your database, and our data preparation module provides just the utilities (not infrastructure) to automate your data pipeline process.

Do take some time to check us out! Quote HN and we give you an additional one day of free trial! :P

<comment><USER2> > Hello! Redash is great if you are looking to host it in-house and have the > engineering resources to set-up and maintain it. If you are looking for an > affordable SaaS alternative...

Well, actually there is a hosted SaaS version of Redash too ([https://redash.io](https://redash.io)). This is what sponsors the work on Redash.

</comment></comment></comment><comment><USER18> For those of you looking for a another fully hosted solution, check out [https://www.cluvio.com](https://www.cluvio.com) (full disclosure: I am one of the founders).

Like redash, Cluvio allows you to run SQL queries against your database and quickly visualize results as beautiful, interactive dashboards, which can easily be shared within your company or externally.

We developed our own custom grammar on top of SQL which makes writing time- range related queries a lot easier and allows to parametrize queries, which powers the dashboard interactivity.

We also allow to run custom R script on top of the SQL results, have SQL Alerts that run at specified schedules, allow you to create SQL Snippets and offer a free entry plan.

Currently supported datasources are Postgres, Redshift, MySQL, MariaDB and Amazon Aurora.

<comment><USER19> Hey Maarius, thanks for chiming in! I've had my eye on Cluvio for a while so I was gonna drop it in here to ask how the two compare.

For me, an open source core is a huge gesture of trust though. It's the ultimate "no lock-in strategy" guarantee. This business model has been proven sustainable by several SAAS companies, such as Automattic (WordPress.com), Discourse (which I work for), Sentry, Piwik and hopefully Redash well!

Any chance Cluvio will consider going down this path too?

<comment><USER2> At the moment (and I hope to keep it this way), it's not only the core of Redash that is open source -- it's the whole product. There are a few features that are introduced on the hosted version first, for ease of development. But most of it goes to the open source first.

As we (Redash) use Discourse as well, it will be only fitting, to have Discourse use Redash :-)

(btw, Sentry are Redash users as well[1])

Feel free to ping me (arik at redash.io).

[1] [https://news.ycombinator.com/item?id=13599631](https://news.ycombinator.com/item?id=13599631)

</comment></comment></comment><comment><USER20> Hello, it seems that there are no instructions on step-by-step installing redash in your own server. The only thing I could find was info on how to install it on AWS or using docker ([https://redash.io/help- onpremise/setup/setting-up-redash-ins...](https://redash.io/help- onpremise/setup/setting-up-redash-instance.html)). However I want to install it normally, using my server's nginx / postgresql / python / supervisorctl etc. Can it be done? Can you provide some step by step instructions ?

Thanks !

<comment><USER2> (Redash author)

Step by step instructions could be great, but there is that much time in the day, so figured that anyone who wants to customize the deployment, will know how do it on his own (or use the bootstrap script for reference).

I'll be happy to answer questions on how to set it up in your environment, you're welcome to the forum[1], Gitter[2] or Slack[3].

[1] [https://discuss.redash.io](https://discuss.redash.io) [2] [https://gitter.im/getredash/redash](https://gitter.im/getredash/redash) [3] [https://slack.redash.io](https://slack.redash.io)

<comment><USER20> Thank you very much! I'll try using the bootstrap script and come to slack if I encounter difficulties. Best regards!

<comment><USER2> Just don't run the script as is on existing server. It's designed to run on a vanilla server.

<comment><USER20> Of course, I'll just use it as a reference to see the required components and steps to integrate them - thanks again!

</comment></comment></comment></comment></comment><comment><USER21> We're hosting on our own EC2 instance using the docker image. Setup was mostly smooth. We love the integration with Google Apps so our entire organization automatically has access.

We primarily use it to connect to Amazon's Redshift and pull some data out for quick visual analysis. It's a very good combo. We're at the early stages of using it but it seems like a very solid product.

Kudos to the dev!

</comment><comment><USER22> Redash is handy, especially if you want to go from "I have a DB of events" to "here's some handy dashboards" in just a few minutes.

Its filtering options were a little limited, at least as of November (you can't inject anything in to the "where" part of a query).

<comment><USER2> Re. filtering: do you mean something like parameters? Because that's supported.

<comment><USER22> Not exactly. Say I have a table with one row per user and want users for a given country. I could say

SELECT country_code AS country_code::multi-filter, count(1) AS users FROM mytable GROUP BY country_code;

And I will indeed get a dropdown with country codes. But, the query is still running for all countries (and if I'm not mistaken all the filtering is on the client). For something more complex, like number of users from a country with spend over a certain amount, I want something like:

SELECT country_code AS "cc::filter", COUNT(1) as num_users FROM mytable WHERE lifetime_spend > {Value from a superset dropdown} GROUP BY country_code

I could do something like

SELECT country_code AS "cc::filter", COUNT(1) as num_users, lifetime_spend from .....

and then filter on the Redash side, I suppose, but that will be slower and have to deal with a lot of data I don't care about.

Has something like this been added in the last few months?

Also, Redash generally seemed to be based around the idea that you would write SQL with a few parameters in the SELECT statement and users would be content with that (which of course is true for many use cases!) We wanted to be able to add a table to Redash/Superset/Metabase/whatever and have someone who doesn't know SQL gain some insights quickly. For instance, distinct users in the last week on a given platform from a set of access logs.

Still though, I meant what I said - I had Redash up and hooked to our DB with useful charts in about 5 minutes, which was spectacular. As time went on though we realized that a lot of people could answer their questions in Superset without asking someone else to write SQL, while Redash required someone to write a query.

Superset is definitely the less mature and polished product, though, perhaps due to its ambition!

<comment><USER2> The "::filter" convention is filters. We also have parameters (for some time now, although only in v0.11 added UI for them). With parameters you can do:

SELECT country_code AS "cc::filter", COUNT(1) as num_users FROM mytable WHERE lifetime_spend > {{minimum_lifetime_spend}} GROUP BY country_code

And minimum_lifetime_spend will render as an input box. Currently we only support input boxes (of different types - number/string/date), but there is a plan to add support for dropdowns there as well.

Allowing self serving without knowing SQL is the goal, but I believe it will take time to do it right. We focus on delivering a great product for people who know SQL, and allow them to give more interactive result sets to other users using things like parameters.

So not sure if our goal is less ambitious, but we just take a different path. And that's good I guess, it would've been not interesting if everyone built the same thing, the same way :-)

<comment><USER22> That's great news about parameters, I don't think I used a version with those included. Thanks again for building a quality tool!

</comment></comment></comment></comment></comment><comment><USER23> Also check out Zeppelin :

"A web-based notebook that enables interactive data analytics. You can make beautiful data-driven, interactive and collaborative documents with SQL, Scala and more."

[https://zeppelin.apache.org/](https://zeppelin.apache.org/)

</comment><comment><USER24> Added the missing Teradata support as the source (kind of) - [https://discuss.redash.io/t/teradata-data-source- support/83/...](https://discuss.redash.io/t/teradata-data-source- support/83/16)

Waiting to resolve few more --

1) Functionality for CSV/XLS file import , like in Tableau . Some discussion on - [https://discuss.redash.io/t/support-csv-data- source/216/5](https://discuss.redash.io/t/support-csv-data-source/216/5)

2) Functionality to select "ALL" in filters

3) Functionality to merge data from multiple data source. I know it does exists in Paid version but waiting for it come in Opensource version.

</comment><comment><USER25> Are some public datasets available directly as SQL endpoints? I know the Semantic Web provides some public datasets as SPARQL endpoints. But I have never heard of an equivalent for SQL.

<comment><USER26> How about the BigQuery ones?

[http://demo.redash.io/](http://demo.redash.io/) works great with BigQuery.

</comment></comment><comment><USER27> Great work! A quick question: what differentiates Redash from Chart.io, Periscope, or existing SQL-and-visualization BI tools?

</comment><comment><USER28> I was working for a company that relied heavily on Wagon - and then it got bought by Box and they shutdown the product. We looked for alternatives but nothing seemed to offer what Wagon had - this looks like it might be a good fit though. Excited to try it out.

<comment><USER29> You should also check out DataMill ([http://www.datamillapp.com](http://www.datamillapp.com)), it's quite similar to what Wagon offered. Installed as an app, context-aware SQL autocomplete and built-in charting. It's in public beta - give it a try :-)

(DataMill author)

</comment></comment><comment><USER30> Ask HN: Is there any open source BI tool that is always first on the list when the conversation finally gets to

 _" Oh, you're on the Windows/Microsoft stack? Then you're probably going to wind up using _______"_

<comment><USER31> I think part of the reason this happens is Java has wider support out of the box for most data sources via JDBC than Windows has with ODBC/ADO.NET. I mean, it's a close race, but most new open source databases have Java/JDBC drivers first and then ODBC second, for example. I think this may change a little with .NET being opened and cross platform, but it might take a while.

Of course you can make Java/JDBC connections from Windows to databases but many users somehow find it more difficult and I've noticed an annoying trend among large enterprises running Windows on the desktop to not want to support Java/JDBC tools on the desktop, unless they are invested in it.

<comment><USER30> Thanks for taking the time to respond!

I'm not unstanding why you mentioned Java/JDBC, I'm not aware of any of the mentioned tools (redash,superset,blazer; turns out metabase uses Clojure) using Java. Only redash doesn't list SQL Server support. What am I missing?

<comment><USER31> Well, I meant in the general context of database connectivity. The underlying database connectivity plumbing is almost always a JDBC or ODBC interface when you're talking a regular DBMS [1].

Historically, ODBC was not always as popular on Linux (being a Microsoft standard originally), although that has changed a lot by today. JDBC was usually more associated to open source/Linux BI/database tools (Java being open source in origin itself), which is why for a while, you could connect to more data sources with JDBC drivers/interfaces than ODBC. Writing a JDBC driver is also many times less complicated than writing an ODBC driver. The ODBC spec is old and complicated.. JDBC is still difficult but much less complicated.

Most programming languages can get to either a JDBC or ODBC driver via some mechanism, on pretty much any platform. So today I guess it's hard to say that it matters (ideally it shouldn't), I was just giving my personal take on the historical context that lead to the comment you made - "why doesn't this run in the Microsoft stack?".

All of the tools mentioned (redash, superset, blazer) are relying on the underlying interfaces in ODBC / JDBC for regular databases. If the system is running on Unix and accessing ODBC, it is almost guaranteed to be using UnixODBC to be doing so. Any tool like this (I don't know the internal- specifics of these tools) would probably have built a layer in to abstract away the low-level interface into some mechanism that hopefully makes the use of JDBC or ODBC (or any interface) irrelevant. That's a lot harder than it seems on the face of it. I used to work for a company that made a product that connected / federated data from "any" platform - which is partly what influences my opinion of how difficult it is to wrangle all these interfaces at lower-levels.

EDIT: I think the crux of what I'm saying is, the reason you don't have a tool that pops up when you ask that question is, because of OS platform differences, with Linux/open source usually being the first priority over Windows historically.

[1] Some DBMS provide native web services connectivity which obviates the need for JDBC/ODBC completely.. which is kind of nice.

<comment><USER32> For Superset we use SQLAlchemy and much of the connectivity goes through the DBAPI Python abstraction, and I believe most of the drivers are using native implementations.

<comment><USER31> And there you have it, "turtles all the way down". SQLAlchemy sitting on DBAPI, where you can get to databases primarily through: 1) ODBC/JDBC, 2) ADO or 3) native Python database drivers contributed by the community that speak the wire protocol of the database (usually wire protocols that are open source or openly documented).

These abstractions most programming languages have make the problem mostly go away for developers.

However, sometimes you'll find (as is sometimes the case with SQL Server, for example) that the fastest or most complete/stable database driver is a specific one that is not quite 100% (but could be 99.9995% supported) completely supported. For example, Python gets to SQL Server via ADO (only on Windows) or some driver (usually ODBC) that talks TDS protocol - often FreeTDS (open wire protocol compatible with SQL Server).

We are definitely in a better state today with regards to programming language/framework support for interfaces to databases, but I think the historical context of the original "generic database interface standards" (JDBC/ODBC) is important to understand to know how we got to where we are, and why sometimes people struggle to ask "what BI tool is the first to come to mind on Microsoft/Windows stack"? Don't get me wrong, plenty of Microsoft/Windows supported BI tools exist, and cross platform abstractions like those in Python make it so OS matters much less (but still matters).

</comment></comment></comment></comment></comment></comment><comment><USER33> Redash is great. We've tried several alternatives, including Metabase, and ended up with extensive Redash use. We use the self-hosted version for sharing data between the tech and business sides, for day-to-day monitoring and for ad-hoc visualizations, and are generally very happy with it. It's not perfect - there are some areas, like fine-control over chart customization, query auto-updates (especially queries with parameters) and the ability to queue parallel queries to the same data source which would be nice improvements, but these are nitpicks - highly recommended.

<comment><USER34> I've been looking at metabase myself. What were your main issues with it?

<comment><USER33> The main issue for us was support of JOINs (note that this was a while ago, not sure what's the status now; I did find this relatively recent discussion, though [1])

[1] [http://www.metabase.com/blog/Joins](http://www.metabase.com/blog/Joins)

</comment></comment></comment><comment><USER35> Do any of the dashboards mentioned support exporting to a report?

<comment><USER2> You mean as a PDF or email?

<comment><USER35> Yes, essentially. An export to a file on a given time period would be fine, can deal with the format conversion etc myself. I'm thinking of sending arbitrary reports to management without the need for Jasper etc.

<comment><USER2> Exports to CSV or Excel are already available. In future releases we will add support for full reports with visualizations and everything.

It's actually already possible with some scripting and our API, but I want to add it into the product itself.

<comment><USER31> Also, publish data set as a web service dataset, and now you're into the features offered by "data virtualization" software vendors / tools (things like Teiid or Denodo, for example).

</comment></comment></comment></comment></comment><comment><USER36> Thanks, @arikfr for the project.

We started using redash about 2 years back. We use it for ad-hoc SQL to Redshift and MySQL and most of our business level KPI dashboards are there. One thing I would want to do is add some metabase like segmentation there(preferably that works with joins), so that marketing/sales people can also actively query from it.

</comment><comment><USER37> We use Redash at Sentry and it's become critical for marketing, sales, even engineering.

Open source ftw!

</comment><comment><USER38> Has anyone prioritized adding the ability to modify the HTML of the report the way ModeAnalytics.com does?

I frequently want to map the data directly to highcharts.js when the UI can't get me to the result I want.

</comment><comment><USER39> Reminds me of Blazer [https://github.com/ankane/blazer](https://github.com/ankane/blazer)

</comment><comment><USER40> For data visualization I recently switched from SQL+Excel to Tableau, and I'm never going back.

<comment><USER41> My qualms with Tableau is that they've moved excruciating slowly on MacOS support and that getting started from scratch seemed like a major development effort. We evaluated Tableau against Looker and are now very happy customers of the latter for a year+.

</comment></comment><comment><USER42> How does the externally hosted version connect to databases?

<comment><USER2> (Redash author)

You will need to allow access from our servers to your database. It's obviously a security concern, but we take great measures to ensure the safety of your data.

If that's not an option, you can always run Redash self hosted.

</comment></comment><comment><USER43> After trying pretty much every BI tool we went with [http://modeanalytics.com/](http://modeanalytics.com/) and have been pretty happy!

</comment><comment><USER44> This is awesome!

</comment><comment><USER45> I didn't find 'The Art Of Computer Programming' by Donald Knuth... Must be a mistake.</comment>


Cuckoo hashing - <AUTHOR> http://en.wikipedia.org/wiki/Cuckoo_hashing

<comment><AUTHOR> BTW, more info on cuckoo hashing:

[http://mybiasedcoin.blogspot.com/2007/06/cuckoo-hashing- theo...](http://mybiasedcoin.blogspot.com/2007/06/cuckoo-hashing-theory-and- practice-part.html)

[http://mybiasedcoin.blogspot.com/2008/05/more-robust- cuckoo-...](http://mybiasedcoin.blogspot.com/2008/05/more-robust-cuckoo- hashing-esa-paper.html)

</comment><comment><USER2> Cute trick, but what makes it front-page hacker news?

<comment><USER2> I wasn't being snarky, it's a serious question. Is there something especially relevant about cuckoo hashing that led someone to submit it?

<comment><AUTHOR> I thought it was an interesting algorithm that I hadn't seen anywhere else. I think that is perfectly sufficient justification for submitting it; as for making it to the frontpage, ask the people who modded it up.

<comment><USER2> I found it interesting, too: I was just wondering if there was some relevance that I was missing.</comment></comment></comment></comment>


Ask HN: Client asked about paying for bug fixes - <AUTHOR> I had a client today asked me about why they should be paying for bug fixes when it seems like it should've been written correctly the first time. They were very understanding with the answer that I gave but I'm curious to know how others have dealt with it in other projects. <comment><USER2> Unless you have a detailed requirements document, documented Test plan and a full Requirements Traceability Matrix (RTM), the client cannot really argue this. A lot of clients will not be able to tell the different between a "bug" and an "enhancement". If I had a penny for every time a client said "System does not work. This is an issue".....Well it is not an "issue". It is a functionality that doesn't exist as per the project scope and now needs to be implemented as a "Change Request" or "enhancement".

</comment><comment><USER3> If it's a time-based contract, then testing and bug-fixing is part of the billable time - it's a normal part of development. The client can decide how much testing and bug-fixing they want, often there is a point where the software is considered good enough to stop spending more expense on it.

For fixed-fee contracts, I usually include testing and bug-fixing in the quoted price, and make sure to include the time spent on that in the estimates. There has to be a sign-off time limit though (for example 1-2 months), during which the client is supposed to test the software to their satisfaction and after which newly discovered bugs are not included - otherwise a project can linger on forever in your schedule.

However with fixed-fee contracts the main difficulty is agreeing on what are bugs, and what are change requests if the initial plan was not detailed enough.

</comment><comment><USER4> My answer is always this. If I write a line of code, discover a bug, and fix it right away, it's part of the normal coding process and covered by the hourly/daily rate I've quoted the client. What difference does it make that the bug was found later or by another person?

</comment><comment><USER5> A good response would be, but "If it wasn't in the requirements, it didn't get coded."

We have to use that response alot when someone asks about a program that doesn't work the way they expect it to. Did you explicitly say that it needs to do this in the contract? No? Then it's not a bug, it's an enhancement.

</comment><comment><USER6> Charge extra up front to write tests. The client gets higher quality code, and generally with less fixes needed.

That being said, there will always be issues until we have AI that writes AI that writes code (with the humans out of the loop).

</comment><comment><USER7> I don't charge for bug fixes and generally try to do high fixed-fee engagements when possible. Hourly billing is hard to get right.</comment>


How to Hack an Election - <AUTHOR> http://www.bloomberg.com/features/2016-how-to-hack-an-election/ <comment><USER2> This shouldn't be a surprise: I think most HN members know that it's possible; we know that the rewards (incredible power, status, and everything you can get with them includig the spoils of office) are worth the investment (only $600K for this solution!); we know that there are many people, including politicians, unsavory enough to do it.

The question is, why do we pretend it's not happening until there's a smoking gun?

Consider the U.S. presidential election: The outcome is worth billions each to very many parties worldwide; it's existential to some foreign governments and actors; many in the U.S. have very powerful ideological motives. How much would it cost to hack voting machines? It seems very likely to me that it's happening, though I don't know where or to what extent.

I know I'm not the first to point out this risk, but why should we assume that the reported vote count represents the actual vote? Becasue there is no smoking gun yet? That's like assuming your critical network hasn't been hacked, and doing nothing to protect it, because you haven't seen evidence of an attack yet.

<comment><USER3> A few reasons I can think of:

1) Hacking voting machines to a significant extent would require a team of people, not just one guy, which dramatically increases the risk of exposure.

2) The reported vote count is usually very close to the predicted vote count, and always very close to exit polling results. The hacked results would need to be very close to the real results, or be combined with control over a huge number of people in the polling industry. Campaigns also run internal polls, so opponent campaigns might become suspicious if their own polls were not matching the results.

3) Recounts are possible and somewhat common. Therefore hacking could not be used to create a very close result that could trigger a recount, or it would need to be combined with actual ballot theft, or control over the officials doing the recount.

These are not theoretical concerns. See Bush v. Gore in Florida for what might happen if someone tried to hack a close election.

It seems to me that hacking an election would only be feasible in an already close race, would require the cooperation of a number of people with huge risks of exposure, and could still be caught by one of the many safeguards that exist.

Obviously I am only talking about hacking voting machines. It would not surprise me if the tactics in this article were common; using Twitter bots, spying on other campaigns, etc.

<comment><USER4> None of this squares with what I have previously been told about US elections.

1\. I have seen video demonstrations wherein one person with opportunistic unsupervised access to a voting machine can hack it to affect every issue on the ballot within five minutes. Given the security at many polling places in the US, this easily extrapolates to one person able to hack all machines at a given polling place prior to election day, and possibly repeat the effort at several other nearby polling places within counties using the same type of machine.

Other hacks target the central tabulators, and don't require touching the voting machines at all. Opportunistic timing to enjoy unsupervised access between the election and results certification is a bit more critical to these.

2\. Exit polls have frequently been "adjusted" in the US to more closely match the actual elections results. In the UK, this is called the Shy Tory Factor[0] or Shy Labour Factor, and has existed since 1992. The US has had the Tom Bradley Effect[1] since 1982.

This is usually explained by presuming that the exit polling result is inaccurate. It is never seriously considered in public that vote fraud may be becoming more severe. Yet I saw a statistical analysis of actual voting results, differentiated by the model of voting machine, that showed a clear, obvious bias towards certain choices of specific issues. The poll adjustment factor can easily be explained by the hypothesis that people remain as honest as ever in exit polls, but the voting process itself has become less honest.

3\. Recounts will not uncover fraud if the records necessary for the recount are subject to the same controls as the records used for the first official count. If the voting machine does not leave a non-electronic record that can be verified by the voter while they are still at the polling place, a recount will just be run on whatever records exist, which may be falsified.

4\. Thanks to the winner-take-all elections in the US, and the availability of previous census and elections data, it is possible to identify specific counties as keystones, wherein efforts to sway the vote locally will have disproportionately effective results.

For instance, you examine the "battleground counties" in the "swing states", and hack the central tabulator in that county to give your favorite candidate a 3% advantage by switching an opposing vote 1.5% of the time. You don't tell your party you're doing it. You don't even tell your _dog_ that you're doing it. You just do it. If your guy loses, well, you tried.

Multiply by dozens of technically competent yet unconnected and independent supporters, and you get a significant extent that does not require any team.

[0] [https://en.wikipedia.org/wiki/Shy_Tory_Factor](https://en.wikipedia.org/wiki/Shy_Tory_Factor)

[1] [https://en.wikipedia.org/wiki/Bradley_effect](https://en.wikipedia.org/wiki/Bradley_effect)

<comment><USER5> I still find it odd that we can receive carbon copies of time cards and receipts for ATM visits (..on thermal paper which decays rapidly), yet when it comes to voting for our state elections we have no record of our vote and everything is handed off to 3rd parties with seemingly no provable accountability or transparency.

      "It's not the votes that count, it's who counts the votes."

<comment><USER6> Eliminating the ultimate privacy of the voting booth has potentially serious negative consequences, like attempts to influence elections via intimidation or bribery ("Show us your vote for Smith and it's worth $100 to you. If you don't, remember we know where you live...") There are a variety of clever schemes that work around this, but none of them are as simple as you getting to bring home a certified copy of your ballot.

<comment><USER7> I would sooner welcome a president distributing $6.5 billion dollars ($100 * winning vote count 2012) to normal people than have to suspend rationality and instead have faith in every actor capable of manipulating votes (of which I feel there are many).

If you honestly believe there is nothing any candidate can offer you which is worth more than $100, I propose you should be allowed to take that deal.

( Of course, completely open source hardware and software is the real solution. )

EDIT: I am a little bitter because I'm in a state using machines which don't even keep a paper trail to begin with. [https://www.verifiedvoting.org/verifier/](https://www.verifiedvoting.org/verifier/)

</comment></comment></comment></comment></comment></comment><comment><USER8> > _He also splurged on the very best fake Twitter profiles; they'd been > maintained for at least a year, giving them a patina of believability._

> _Sepúlveda managed thousands of such fake profiles and used the accounts to > shape discussion around topics such as Peña Nieto's plan to end drug > violence, priming the social media pump with views that real users would > mimic. For less nuanced work, he had a larger army of 30,000 Twitter bots, > automatic posters that could create trends. One conversation he started > stoked fear that the more López Obrador rose in the polls, the lower the > peso would sink. Sepúlveda knew the currency issue was a major > vulnerability; he'd read it in the candidate's own internal staff memos._

I've been wondering when more sophisticated Twitter sockpuppetry would start having an impact. The media seems generally unprepared for a coordinated campaign of fake tweets from accounts that don't have eggs as their avatar or #freeiphones in their profiles...I don't mean just the trend of "Hey 3 people said something on Twitter so it's a story", but that there's also not enough skepticism paired with efficient ways for sniffing out fakery on social media. I hadn't thought about the problem of a mass army of bots suddenly creating a trending tag.

It reminds me of "The Agency", a story published last year investigating how Russian operatives allegedly used human-controlled Twitter spamming to try to publicize a fake story of a Louisiana chemical plant explosion:

[http://www.nytimes.com/2015/06/07/magazine/the- agency.html?_...](http://www.nytimes.com/2015/06/07/magazine/the- agency.html?_r=0)

But as sophisticated as that sockpuppet campaign was...it was obviously destined to fail by the fact that an exploding chemical plant is fairly easy to confirm as true or false in real life. Trending opinions about politics don't offer such binary ways of filtering.

<comment> rm_-rf_slash Propaganda takes time. Often the most effective stuff happens when you aren't even aware of it. You progress through your regular life and encounter regular things. People bitch on the internet about stupid stuff with the occasional decent argument, and nothing looks out of the ordinary. You see laughable arguments about politicians with hints of truth, or intrude, but maybe you ignore it.

After a while you see more arguments along the same line of reasoning, but with more data and less vitriol. You find yourself agreeing more, even if you like the person being disparaged. You may even come around to accept a side of an argument you didn't think much about before.

This is how minds are turned. First the trolls and bots do as they do. Some people catch a whiff and continue the trend. Sometimes as a joke, other times seriously. Even those who pretend to be joking will not long be indistinguishable from the fools who think they are in good company.

Propaganda is not a tidal wave that comes crashing down on established opinions, it is a steady stream that eats at solid rock until it creates a deep gorge you could not have imagined structured in any other way.

<comment><USER9> The world is neck deep in dearly-held beliefs that would quickly die out if people en-masse embraced the kind of systematic, relentless skepticism required to combat this. Society protects these beliefs by stigmatizing skepticism, but in doing so effectively makes itself vulnerable to exploit.

<comment> rm_-rf_slash I myself am skeptical about that.

Edward Bernays, who really pioneered propaganda, argued that people put elements of identity in "logic-proof compartments" because to be one thing by definition means not being another thing, and people often would rather shut out thoughts than seriously question themselves.

Are you an "entrepreneur?" What about "talented?" Or "Democrat?" Or even, "Liberal?" "Conservative?"

Public opinion is as much an influence on propagandists as they are on public opinion.

</comment></comment></comment></comment><comment><USER10> > with a shaved head, goatee, and a tattoo of a QR code containing an > encryption key on the back of his head. On his nape are the words "</head>" > and "<body>" stacked atop each other, dark riffs on coding.

Jesus, this guy sounds like he walked straight out of a bad cyberpunk story.

On the serious side, this gets to me:

> Sepúlveda says many of the candidates he helped might not even have known > about his role; he says he met only a few.

It never occurred to me, but it's a novel concept: people campaigning for high office may be unaware of corruption in their own campaign. The idea of a well- intentioned politician with no idea corruption propelled them into office is scarier to me than a knowingly corrupt politician.

<comment><USER11> It is just plausible dependability for the candidate. This is traditionally how organized crime works.

<comment><USER12> I believe it is called "plausible deniability". ;)

<comment><USER11> Yay! I'm not the only one old enough to have seen that movie on HN.

</comment></comment></comment></comment><comment><USER13> Capitalism makes centralized democracy untenable, and capitalism is here to stay.

So much power is controlled by infrequent elections, so influencing those elections is always worthwhile. Capitalism is an evolutionary force that selects for the most profitable business practices. People who figure out how to profit from influencing our democracies will outcompete those who don't. We can pass new laws, but that's like inventing a new antibiotic: it works for a while until evolution finds a way.

We must decentralize governance to be free. But how? We can use trade to individually enforce rules. Consider ISIS. ISIS controls territory by paying people to control territory. The money ISIS pays is worthless paper. You and I give that paper value because we accept it for our work. If you want to reduce ISIS's power without anyone's permission, all you have to do is stop accepting any money they've traded for your work.

We now have the ability to build decentralized ledgers of trade that you can consult to do this. Instead of relying on political parties to pursue your policy goals, you'll install an app that cuts people off from your economy when they break your rules. If this is a power that people desire, they must stop accepting anonymous money and only accept money through their policy wallet apps.

We're using bad technology to organize our society and it's giving us political corruption, terrorism, and global warming. It's imperative that we decentralize democracy so we can rid the world of this unnecessary damage.

<comment><USER14> As an alternative, what about moving away from representative democracy (i.e. where you periodically elect someone to make decisions for you by proxy), to a system where you can optionally choose to cast your own vote for particular issues. If we had more lightweight mechanisms for performing trustworthy elections then maybe this would be feasible. I imagine by default though most people wouldn't be bothered to put in the effort to make knowledgeable decisions, and hence why you want to still have representatives.

<comment><USER13> "Liquid democracy" is very cool, and it seems strictly superior to our current system. However, it will still be captured by capitalism. The most popular representatives will attract the most funding, and their actions will be as biased toward their funders as they can be without losing too much support. The big threat there is regulatory capture: promoting laws to clamp down on profitable industries, but in ways that give incumbents huge advantages because the laws are hard to comply with.

The only way to get money out of politics is to make sure the people can control money.

<comment><USER14> Sure but at least it reduces, albeit doesn't eliminate, the ability of politicians to say one thing at election time and then do the opposite once they've been elected.

</comment></comment></comment></comment><comment><USER15> Computers are flexible mutable controllable interconnected facile adaptable devices - it's why we use them and love them so. What is it we hope for as the basic property to vote collection and reporting? It's essentially the opposite of the computer's character set. So if you insist on the convenience advances to the voting process, ("I wanna be able to vote via smart-phone!") then you'll inevitably be giving up the basic property that you really do want: veracity. So, to the first approximation: primitive paper ballots are some of the most stubborn and costly to hack.

<comment><USER16> Keep in mind that you can give up anonymity instead, via cryptographically verifiable voting with confirmation receipts ;) Of course, that opens its own can of worms...

</comment></comment><comment><USER12> oh please. He didn't hack an election. They tried to influence public sentiment using social media. They hacked some phones and emails. But they didn't rigged the election.

Just so you know. The Mexican electoral system looks like it was built by paranoiac maniac. It has many locks. Everything is done manually, votes are counted by multiple citizens, they use transparent poll boxes, special election photo ids are issued for every citizen, even the voting lists have photos, people get their thumb marked with ink, the parties are allowed to have representatives at every place, results at every place are signed, they are published at the voting place and also electronically, etc, etc. One million people participate on the election.

</comment><comment><USER17> Clint Curtis testified before congress that he did just this.

It remains an unproven allegation.

[https://www.youtube.com/watch?v=3YKpvTBmdCI](https://www.youtube.com/watch?v=3YKpvTBmdCI)

[https://en.wikipedia.org/wiki/Clint_Curtis](https://en.wikipedia.org/wiki/Clint_Curtis)

</comment><comment><USER18> Somewhat related story: During the Mexican presidential election mentioned in the article, a PhD student at one of the top Computer Science departments in the U.S. got in touch with me (through a shared friend) because he saw I was doing research on Mexican civic tech.

He mentioned that while he was teaching a workshop at his university, two of the workshop participants, who were employees of a powerful Mexican TV network, asked him to help them discredit or deflect the student movement described in the article (the movement was called the "Mexican spring" at the time for their use of social media).

This sounded shady to the student, and he wanted to tell someone, but didn't know who. Ultimately he just avoided the TV folks for the rest of the workshop.

Weeks later, The Guardian published some leaked documents that showed nefarious ties between said TV network and the then candidate.

The student connected with the press then, but the story never came out. Perhaps he got cold feet, or maybe the journalists didn't think it was interesting enough.

</comment><comment><USER19> Ugh. I don't know if the particular people involved here are telling the truth, but I basically know that something like this must be happening, given the incentives involved. Dispiriting, I have to say, since it would be hard to beat this sort of thing without becoming what you're fighting against.

<comment><USER19> Also, at the end there when they implied that HRC is hiring Rendon for the general election, did anyone else find themselves thinking that this whole story could be a manipulative plant of the very sort they were discussing? Not that I think HRC is above this sort of thing.

<comment><USER20> I wonder if the implication was that the candidate from Texas was the interested party instead of HRC.

I know she is a "leading candidate" and no party affiliation was mentioned but the hacker has in the past supported right-wing candidates and Ted Cruz is also closer to being the annointed one than he was a month ago. It might not be a coincidence that dirty tricks reminiscent of past elections are just starting.

It certainly appears that he is already on someone's payroll. Trump? Maybe, though it would be denied of course. Maybe this is Trump's answer to Cruz's manager.

It appears though that Cruz already has a slimer helping him according to a story I read a few weeks ago.

Maybe it is HRC and she's just trying to gear up for the mud-fest that will surely start once the real campaign kicks off.

[Behind Ted Cruz's Campaign Manager, Scorched Earth and Election Victories - NYTimes]([http://www.nytimes.com/2016/02/24/us/politics/ted-cruz- campa...](http://www.nytimes.com/2016/02/24/us/politics/ted-cruz-campaign- manager-jeff-roe.html?_r=0))

<comment><USER21> It was Rendón, not Sepúlveda (the hacker, in prison), who was allegedly helping Trump.

      > But Rendón says he's in talks with another leading
      U.S. presidential campaign—he wouldn't say which—to
      begin working for it once the primaries wrap up and the
      general election begins.

<comment><USER20> That's right.

I didn't keep the two of them clearly separated in my mind since they had operated as a team for so long.

I wonder though if part of his access to computers while locked up allows him opportunity to carry on the same sort of operations as before, this time only for "approved" clients.

Elections aren't what they used to be. Maybe they never were.

</comment></comment></comment></comment></comment><comment><USER22> No open <head> and no close </body> ... I don't trust this guy.

</comment><comment><USER23> This is scary. Although we believe that an intelligent person would probably distinguish most social media manipulations, I think that we fall into the trap of deceptions more than we think we do. I wonder if there is some sort of study on how often significan social media manipulations occure in elections on the west.

</comment><comment><USER24> Reminds me of a meeting I sat some years back with someone whose role was to run 'dirty tricks' campaigns for Microsoft in the early Apple vs. Microsoft days. He talked about stuff like ways they used to undermine Apple dev conferences so no-one would be interested to go the following year and generally white-anting their community, especially education environment. It was a bit of an eye opener.

I also suspect this guy is underselling his service. The article calls $20k/mth expensive. If your going to significantly affect an election I suspect there could be another zero or more there. And I wonder if he's ever worked for 2 sides simultaneously? Given the ethics involved this wouldn't be a stretch.

</comment><comment><USER25> Hi guys, I'm a journalist from the Netherlands and I'm working on this story. Basically in a way that you comment on it, asking myself the question what this story means for more adult democracies. So my question for you is: do you think it's likely that political campaigns have hackers on a payroll to spy on opponents agenda's and manipulate social media? I'd like to hear from you. You can email me as well on firstname.lastname@example.com

</comment><comment><USER26> There are many comments here pointing out the (quite reasonable) fact that hacking voting machines on a large scale is likely to be discovered and thus unsuccessful in stealing a U.S. Presidential election.

Even so, the average voter is quite protective of the idea that American elections are structurally fair (a sentiment that is actively inflamed to imposed voter disenfranchisement schemes, like ID requirements).

If the goal is to expose the fundamentally insecure nature of electronic voting in the US (and potential for truly fraudulent election results in tight districts) a complete invalidation of voter ballots [i.e. Big splashy, obviously fraudulent election results] would surely draw enough attention to force some public discussion of investing in secure, auditable voting machines.

</comment><comment><USER27> I found this article very interesting. I do always wonder why we pretend this isn't happening though? That's not ignorance, that's just plain stupid.

</comment><comment><USER28> Fixing the results 49% to 51%, is a way for them to seem plausible.

American Election Hacker Testifies [https://www.youtube.com/watch?v=DzBI33kOiKc](https://www.youtube.com/watch?v=DzBI33kOiKc)

</comment><comment><USER29> Am I the only one who misread this as "How to Hack an Elect_r_on"? :)

</comment><comment><USER30> More interesting for me is why hacker decided to share it all with press agency. Is this information in the article? I might have missed it. Seems like a good way to get yourself kidnapped and tortured indefinitely.

</comment><comment><USER31> Why Electronic Voting is a BAD Idea - Computerphile: [https://www.youtube.com/watch?v=w3_0x6oaDmI](https://www.youtube.com/watch?v=w3_0x6oaDmI)

</comment><comment><USER32> A 600k budget to fix or change the results of an election for a country with a GDP of 1.2 trillion?

That's a ridiculously good use of money for a corrupt politician.

</comment><comment><USER33> Exceptionally interesting.

Also, First HN article I've also seen on Drudge. Worlds collide.</comment>


Ask HN: Why are resumes a problem? - <AUTHOR> A bunch of companies (even small ones) have chosen to use forms and text extractors on resumes during their application process. I'm not convinced that this is solving a real problem.<p>why is this a problem? what specifically does this solve? If you use these services, do you find them better than receiving a resume? <comment><USER2> They solve the problem of having to manually go through 1000s of applicants. Most jobs these days have over 100 people applying. This software allows the system to easily get rid of unqualified (whatever you set the criteria at) applicants automatically.

I am sure there are many other uses like applicant tracking, job posting, compliance with discrimination laws, etc.</comment>


WTF Am I Looking At? How Not to Send a Mail - <AUTHOR> https://medium.com/@barakkassar/wtf-how-not-to-send-a-mail-75623472c868#.i9st5ob4m <comment><USER2> On this note, I read the article and thought maybe I should feel like "WTF Am I Looking At? How Not to Write an Article".

The title of the paragraph that says "The Problem" says this:

> Do I need a new account? "Get a New Account" is the the only obvious button > I'm prompted to click. Yet, when I click it nothing becomes any clearer.

That is not a clear enough problem statement for some random person on the Internet to understand what the author is even talking about at first read! Instead of a long winded article (which Medium as a platform seems to attract), where the problem is explained in terms of the solution, and which still didn't seem clear by the end, the problem could've been explained better initially.

Edit: The article about "Tappers and listeners" that was linked to was great!

<comment><AUTHOR> fair point!

</comment></comment><comment><USER3> Is it really that common to refer to email as mail? I had a different imagination when I clicked on the title...

<comment><AUTHOR> i did think about that when writing the headline and post. i hope you don't feel bait-and-switched. it seemed to me that in context of web the e was not necessary. next time i'll consider putting back the e.

<comment><USER3> No, the title rang a bell because I was recently concerned about sending my Passport over mail to an embassy for VISA. So I wanted to know if there was something crazy about mail :-)

The post interesting nevertheless. I've had my share of similar email experiences as well.

<comment><USER4> Just FYI, the credit/debit card company is often stylized as "VISA", and the travel document is a "visa".

<comment><USER3> True. I stand educated and corrected. [can't edit the post anymore though]

</comment></comment></comment></comment></comment><comment><USER5> How about just don't send spam?

If you really "get into their head" you'll find what they really want is no unsolicited email at all.

</comment><comment><USER6> I work in higher education, which means there are _a lot_ of {terrible} emails sent out daily. Recently I've started educating the senders on how to be more effective, as well as not just copy and paste event listings from our website to Mail Chimp.

It's a challenge teaching people how to be more effective in their emails communications.

</comment><comment><USER7> "Get in their head"

Perfect advise. And if that's not possible: Ask someone in your target audience to give you feedback.

You don't need A/B tests, fancy markup or the newest marketing patterns if the gist of your message is misunderstood.

Ideally, the gist of the message should be in the subject line.</comment>


The Physics of Butterfly Wings - <AUTHOR> https://johncarlosbaez.wordpress.com/2015/08/11/the-physics-of-butterfly-wings/

<comment><USER2> Hopefully one thing this work will lead to is more physically accurate iridescence shaders for 3D render engines. The ones I've seen so far only attempt to fake iridescence at an approximate level, and the results are not very convincing.

<comment><USER3> To simulate iridescence at this level would require a tremendous level of computation that we just do not have yet.</comment></comment>


This Is How Much Money Twitter Owes You - <AUTHOR> http://newsfeed.time.com/2013/11/07/interactive-this-is-how-much-money-twitter-owes-you/

<comment><USER2> This is silly - they'd have to subtract the PR value that Justin Bieber or Katy Perry gets from Twitter, for free or nearly free.</comment>


Dyson's Latest Feat of Over-Engineering: A Filterless Vacuum - <AUTHOR> http://www.wired.com/2015/01/dysons-latest-feat-engineering-filterless-vacuum/ <comment><USER2> Very cool. The next step would be a vacuum that empties itself. I'd opt for incorporating a docking/charging station into a trash can, so it empties every time it goes back to charge, and vice versa. Though since it's Dyson, it'd probably be a sleek auto-open, compacting trashcan.

</comment><comment><USER3> I wonder what kinds of particles are re-introduced to the air between the 0.5 um size this vacuum filters and a traditional HEPA filter which excludes particles down to 0.3 um.</comment>


Amazon Route 53 Reduces Hosted Zone Pricing - <AUTHOR> http://aws.amazon.com/route53/pricing/

<comment><USER2> If you are interested in DNS performance comparison, I created a tool for comparing DNS query times of 16 different hosted DNS providers including Route 53, Dyn, DNS Made Easy, UltraDNS, Rackspace, SoftLayer, Easy DNS, Zerigo and a few others here: <http://cloudharmony.com/dnstest>

It is a browser/javascript based test that uses a wildcard DNS record and alternating between cached and non cached lookups to determine an approximate query time for each provider.

<comment><USER3> I ultimately went with Route 53 because of the performance. Glad to see your tests confirms my choice.

Switching to Route 53 from MediaTemple as my DNS was like a night and day difference. The site felt much faster.

<comment><USER4> I'm a little confused here. Wouldn't client DNS resolution performance really depend on your local DNS cache (e.g. the DNS server at your ISP)? Nobody going to your site will hit Route 53 or MediaTemple's DNS servers directly. These only serve as the source of truth for the thousands of DNS caches out there that clients actually hit.

<comment><USER2> Typically this is true as the only performance hit will be due to the initial lookup, and then your resolver will cache the record. However, if your DNS record uses a low TTL (required for things like DNS-based monitoring/failover) like 30 secs or 1 minute, DNS lookups will have a larger impact on site performance because user resolvers will have to frequently re-query the authoritative DNS.

</comment></comment></comment></comment><comment><USER5> Route53 is an excellent service, and the latest price reduction makes it even more attractive - but using the API directly as an end user is a little cumbersome, especially for cloud servers.

A while ago I thought "wouldn't it be great if when launching a cloud server some magic would happen and the server would get assigned a human/friendly name of my choice? I'm tired of remembering IP addresses, and logging into my DNS management console to setup records."

Following that thought we added Domain management and a free dynamic DNS service to the TurnKey Linux Hub.

If anyone is interested:

<http://www.turnkeylinux.org/blog/hub-domains> (announcement)

<http://www.turnkeylinux.org/docs/hubdns> (documentation)

<https://github.com/turnkeylinux/hubdns> (source code)

</comment><comment><USER6> That is a welcome change. Previously, Route 53 was useless for services that involve hosting user domains (web hosting, for example) because of $1/month charge per zone. 1000 low traffic sites would cost $1000 per month just for domain zones. With the new pricing the cost would be $110. That is close or even lower than what competitors are charging.

For those on HN who use Route 53: Would you recommend switching from another provider?

<comment><USER7> I use route53 for all of my domains _(I don't have anywhere near 1000)_. The $1/mo was worth it to me simply to ease the pain of updating DNS information; I set the name servers once _(to point to AWS)_ , and then only have to look to one place _(through shell!)_ to manage DNS. I switch VPS hosts often enough that having shell based tool like route53 to switch DNS records has come in very handy. I also have a habit of switching domain registrars regularly. Having DNS managed by a 3rd party like route53 makes a change like this simple _(just updating name servers once the transfer's done)_. I don't have the time to learn how to manage my own DNS server, so I'm happy to pay $1/mo to make my life a bit easier, and even happier to pay $0.50.

I use pcorliss' ruby_route_53 gem[1].

[1] <https://github.com/pcorliss/ruby_route_53>

</comment></comment><comment><USER8> We've (Epio) been using Route53 for 6 months or so now, and have had zero problems with it. The main problem is, as others have said, that the only way to interact with it is via the (somewhat cumbersome) API.

We got round this by writing our own Python command-line tool using Boto, which sadly I have yet to release...

<comment><USER9> I've been using the wonderful cli53 (available on GitHub here: <https://github.com/barnybug/cli53>).

All I have to type is "cli53 import example.com --file example.bind --replace --wait" and it'll replace all of the records for example.com with what's in the BIND file, and the --wait makes it poll Amazon until the "pending changes" go through.

<comment><USER8> Ah, now that looks useful - thanks! And it supports AAAA records, thankfully.

</comment></comment></comment><comment><USER10> I really do not understand the issue with DNS.

It is a dead-simple protocol, and the level of caching at various levels between your server and the user would seem to make super-fast DNS serving irrelevant to the browser experience.

As it is, queries on a 2004 level server box running lots of other services are under 2ms on a local network, so really network latency is the issue...

having 2 simple 1U boxes colocated in NYC and LA, would cost under $200 per month and give you full control and the ability to host thousands if not tens of thousands of domains.

<comment><USER11> You're right, it would be cheap to colocate two boxes to run DNS. Unfortunately you still have to:

\- Purchase the hardware

\- Receive the hardware

\- Install an OS on the hardware

\- Harden the OS from intruders

\- Decide how to manage the servers and setup that infrastructure (puppet/chef/cfengine/ssh/etc)

\- Install BIND

\- Load in all your zones

\- Ship the server(s) to each of the colo facilities

\- Pay for smart hands to install/configure the hosts

\- Setup a procedure to reliably sync zones from your RCS to the hosts and safely load zone changes without crashing BIND

\- Monitor the service and hosts

\- Setup alerting and escalations for when there are problems

\- Subscribe to BIND mailing lists and keep up with the community

\- ~once every couple years perform an ASAP update to BIND after a massive security hole is discovered (without getting compromised during this time)

\- Update BIND regularly

\- Deal with occasional outages at facilities

\- Configure anycast routing for your IP's

Route 53 reduces this to:

\- Make an API call

\- Go back to developing your actual product.

<comment><USER10> I use djbdns / tinydns.

So really, I don't have to do any of that.

What you really would do, given your list above, would be

\- Buy hardware from vendor, ship direct to colo.

\- When racked, they hook up a KVM/IP device that has virtual media support to the system.

\- You install your OS from your desktop machine.

\- Turn off all services except DNS and public key SSH with hosts.deny and hosts.allow setup appropriately

\- Use automated scripts to move zones from current setup to djbdns format .

\- Tweak Makefile for djbdns (included in the distribution) to use ssh to move both the datafile and a text-version backup of the file, to each system. You can keep your original files on a machine local to you and push the changes via SSH that way.

\- you should have a machine or two outside your facility to run e.g. smokeping or other measurement/monitoring tools anyways; so run smokeping etc. on the same machine and if you are paranoid, run the webserver on 127.0.0.1 and use SSH tunneling to view the pages.

Usually for $100/month per server, you can get outage monitoring and 30 minutes of remote hands per month, included.

</comment></comment></comment><comment><USER12> It's still almost 3x as much as dnsmadeeasy.com. I pay $60/year to manage up to 25 domains with 10mil queries. Looks like Route 53 would make sense if you need an order of magnitude more queries, though.

</comment><comment><USER13> Has amazon developed an interface for administering Route 53 domains? Relying on 3rd party interfaces has always been the real deal breaker with Route 53 for me.

</comment><comment><USER14> I use CloudFlare to manage my zones for free :/

<comment><USER15> I second that</comment></comment>


Ymagine: a fast native image decoding and processing library - <AUTHOR> https://github.com/yahoo/ygloo-ymagine <comment><USER2> See documentation here: [https://github.com/yahoo/ygloo/wiki/Ymagine](https://github.com/yahoo/ygloo/wiki/Ymagine)

It's an image processing library for Android developers working on Linux or Mac platforms. iOS development may also be supported as well (the project has an iOS directory), but is probably not yet ready for public consumption since I did not see iOS mentioned in the documentation.</comment>


Not a Slippery Slope, but a Jump Off the Cliff - <AUTHOR> https://lawfareblog.com/not-slippery-slope-jump-cliff <comment><USER2> The worry in this case is the troubling precedent it would set.</comment>


George Orwell on Writing Code - <AUTHOR> https://borislav.xyz/observations/george-orwell-on-writing-code/ <comment><USER2> The rules cited come from the essay Politics and the English Language. [i] The style guide of the Economist newspaper is predicated on that essay. [ii]

[i] [https://www.orwell.ru/library/essays/politics/english/e_poli...](https://www.orwell.ru/library/essays/politics/english/e_polit)

[ii] A copy of The Economist style guide at archive.org - [https://archive.org/details/EconomistBooksTheEconomistStyleG...](https://archive.org/details/EconomistBooksTheEconomistStyleGuideTheEconomistPublicAffairs2015/page/n7/mode/2up)

\- the latest edition is sold via The Economist and by some book sellers.

<comment><AUTHOR> Thanks for [ii]! Didn't know that Economist reused the Orwell's wisdom :)</comment></comment>


Shopify, Suddenly Worth $117B, Is One of the Biggest Pandemic Winners - <AUTHOR> https://www.wsj.com/articles/shopify-suddenly-worth-117-billion-is-one-of-the-biggest-pandemic-winners-11599557400 <comment><USER2> Can we officially end the "rails doesn't scale" mantra now?</comment>