A high-speed network driver written in C, Rust, Go, C#, Java - <AUTHOR> https://github.com/ixy-languages/ixy-languages <comment><USER2> There is discussion about C# and Java being faster than Go, but one interesting thing to note is that both C# and Java have to use C to interface with the kernel.

Java: [https://github.com/ixy- languages/ixy.java/blob/master/ixy/sr...](https://github.com/ixy- languages/ixy.java/blob/master/ixy/src/ixy/c/ixy.c)

C#: [https://github.com/ixy- languages/ixy.cs/blob/master/src/ixy_...](https://github.com/ixy- languages/ixy.cs/blob/master/src/ixy_c/ixy.c)

Java needs a bit more C to make it work. C# only seems to need it for DMA access. But when you look at the Go code, they got away with being pure Go and using the syscall and unsafe package. So that's at least one plus for Go.

(the main readme calls this out, but at least one thing worth mentioning here too).

As a Java coder for my day-job, I do like the breakdown they have of the performance of the different GCs for their Java implementation. [https://github.com/ixy-languages/ixy- languages/blob/master/J...](https://github.com/ixy-languages/ixy- languages/blob/master/Java-garbage-collectors.md)

<comment><USER3> I can't really believe the Swift implementation needs to be that slow. Objective-C used to be 100% C compatible and Swift more or less has complete bridging to C because of the need to use these API's.

Objective-C was often called slow because iteration NSArray was much slower than doing it in C. Well, if you needed to do it fast in Objective-C you wouldn't do it using the user friendly and safe (for 1984) higher level objects.

I think only Rust really allows you to write really safe and still really fast code though.

<comment><USER4> Yes, we could write most of the critical part in C and it would probably be faster. But then it wouldn't be a Swift driver.

<comment><USER5> Are these benchmarks single-threaded? I took a brief look at the Swift codebase, and I noticed that you are using semaphores, but there doesn't seem to be any parallel execution anywhere in the project.

<comment><USER4> The Semaphore is only used during initialization, never in the critical path, see profiling results in the main repo

</comment></comment></comment></comment></comment><comment><USER6> If you can't see or interpret the graphs (mobile browser, etc.) here's a quick description of the relative performance in terms that might be useful even without the graphs.

Bidirectional forwarding, Packets per second: Here, the batch size matters; small batches have a lower packet rate across the board. Each language has increasing throughput with increasing batch size up to some point, and then the chart goes flat. Python is by far the slowest, not even diverging from the zero line. C is consistently the fastest, but flattens out at 16-packet batch at 27Mpps. Rust is consistently about 10% slower than C until C flattens out, then Rust catches up at the 32-packet batch size, and both are flat at 27Mpps. Go is every so slightly faster than C# until the 16-packet batch size where they cross (at 19Mpps), then C# is consistently about 2Mpps faster than Go. At the 256-packet batch size, C# reaches 27Mpps, and Go 25Mpps. Java is faster than C# and Go at very low batch sizes, but at 4 packets per batch Java slows down (10Mpps), and quickly reaches its peak of 11 to 12 Mpps. OCaml and Haskell follow a similar curve, with Haskell consistently about 15% slower than Java, and Ocaml somewhere between the two. Finally, Swift and Javascript are indistinguishable from each other, both about half the speed of Haskell across the board.

Latency, at 90, 99, 99.9, 99.99.. etc., percentile. 1Mpps: All have zero-ish latency at the 90 percentile point, then Javascript latency quickly jumps to 150us, then again at 99.99%ile jumps again to 300us. C# is the next to increase: at the 99%ile mark there's a steady increase till it hits 40us at 99.99%ile. Then a steady increase to about 60us. Haskell keeps it at about 10us until 99.99%ile, then a steady increase to about 60us, and a sudden spike at the end to 250us. Java latency remains low until 99.95%ile, then it quickly spikes up reaching a max of 325us. Next OCaml spikes at around 99.99%ile, reaching a max of about 170us. Next comes Swift, with a maximum of about 70us. Finally, C, Rust, and Go have the lowest latency. Rust and C are indistinguishable, and Go latency diverges to about 20% higher than the other two at the 99.999%ile mark, where it sways, eventually hitting around 25us while C and rust hit about 22us.

<comment><USER7> The Rust page also compares the performance of the Rust implementation using prefetching, which slightly outperforms C for some batch sizes. [https://github.com/ixy-languages/ixy.rs#performance](https://github.com/ixy- languages/ixy.rs#performance)

It would be a bit of a cheat, as it isn't portable, but it would be nice to see prefetching in the C implementation for the sake of comparison.

</comment></comment><comment><USER8> Cross-language comparisons are always interesting to look at; if I had the time, I'd really like to write one in Asm and see how it compares.

I've written NIC drivers for some older chipsets, and IMHO it's not something that's particularly "algorithmic" in computation or could necessarily show off/exercise a programming language well; what's really measured here is probably an approximation to how fast these languages can copy memory, because that's ultimately what a NIC driver mostly does (besides waiting.) To send, you put the data in a buffer and tell the NIC to send it. To receive, the NIC tells you when it has received something, and you copy the data out. Nonetheless, the astonishingly bad performance of the Python version is surprising.

Although I haven't looked at the source in any detail, I know that newer NICs do a lot more of the processing (e.g. checksums) that would've been done in the host software, so that would be another way in which the performance of the host software wouldn't be evident.

One other thing I'd like to see is a chart of the binary sizes too (with and without all the runtime dependencies).

<comment><USER9> > Nonetheless, the astonishingly bad performance of the Python version is > surprising.

In the paper, they point out that the Python version is the only one they didn't bother to optimize.

However, my takeaway is that practically everybody can handle north of 1 Gigabits per second (2 Million packets per second x 64 bytes per packet) even on a 1.6GHz core. I find _THAT_ quite a bit more astonishing actually.

<comment><USER10> I don't see why it's that surprising. We've been stuck on 1Gbps for the better part of 20 years. What's surprising to me is that wired networking was sorta left behind the tech wave, sure 10Gbps exists but it's still not that affordable or widespread.

<comment><USER11> I wouldn't say it was exactly left behind, because the average consumer will not really benefit from anything over 1Gbit. 1Gbit is already enough to saturate most consumer harddrives.

I run 10gbit inside my home and it didn't even cost me that much (if you go with 10Gbit fiber instead of copper) with the sole reasons of getting quicker transfers between my PC and NAS. My NAS has 4 SFP+ ports and functions as a switch. I bought second hand PCIe SFP+ NICs for $40 each and matching transceivers for $15 each. 10M of fiber costs less than $10.

There's no point in going higher, because 10Gbit is already way past the sequential writing speed of the drive array in my NAS, and it's pretty much saturating the NVMe cache drive in the NAS or the NVMe storage in my PC.

That's not to say you can't go faster, because 100, 200 and 400Gbit are very much possible and in use in datacenters and the like.

<comment><USER12> > I wouldn't say it was exactly left behind, because the average consumer will > not really benefit from anything over 1Gbit. 1Gbit is already enough to > saturate most consumer harddrives.

That hasn't been true for a long time. Even one single spinning rust hard drive made in the last decade can do sequential reads at ~120-150MiB/sec, which is easily enough to saturate a 1 Gbit/s link.

SSDs have way, way higher throughput for sequential read and write. Good SSDs will also beat that number handily for random read/writes.

And of course, any machine with more than 1 hard drive can easily saturate a 1Gbit/s network.

I also find it surprising that wired networking has been 'stuck' on 1Gbit/s for decades.

</comment></comment></comment></comment></comment><comment><USER13> That JavaScript and Swift have essentially the same performance here is extremely telling: there are essentially four performance regimes (five if you count Python, but clearly from the graphs you should not ;P), and what would really be interesting--and which this page isn't bothering to even examine?! :(--is what is causing each of these four regimes. I want to know what is so similar about C# and Go that is causing them to have about the same performance, and yet much more performance (at higher batch sizes) than the regime of Java/OCaml/Haskell (a group which can't be explained by their garbage collectors as one of the garbage collectors tested for Java was "don't collect garbage" and it had the same performance). It frankly makes me expect there to be some algorithmic difference between those two regimes that is causing the difference, and it has nothing to do with language/runtime/fundamental performance.

<comment><USER4> Swift spends 76% of the time incrementing/decrementing reference counts; ARC is just very bad at pushing tens of millions of objects through it every second.

There's some more evlauation for Swift here: [https://github.com/ixy- languages/ixy.swift/tree/master/perfo...](https://github.com/ixy- languages/ixy.swift/tree/master/performance)

It's just a coincidence that JavaScript and Swift end up with almost the same performance; there is nothing similar between these two runtimes and implementations.

<comment><USER5> This is also a clear optimization target. It is very possible to write Swift code which requires very little reference-counting overhead.

<comment><USER3> The problem is that 99% of all Swift developers use the language to create front-ends for powerful devices and you never need to squeeze the last drop of performance out of them.

</comment></comment></comment></comment><comment><USER14> The author of this project presented it last year at CCC, here is the video: [https://media.ccc.de/v/35c3-9670-safe_and_secure_drivers_in_...](https://media.ccc.de/v/35c3-9670-safe_and_secure_drivers_in_high- level_languages)

<comment><USER15> Thanks, that was interesting. If anyone is excited enough to try driving peripherals in userspace via hardware registers, I can recommend starting with a Raspberry Pi, since it has several well documented peripherals (UART, SPI, I2C, DMA, and of course lots of GPIO), and the techniques described in this talk are transferable.

A search for 'raspberry pi mmap' will yield a lot of good starting points.

</comment></comment><comment><USER16> Cool to see C# being up there close to C and before Golang.

I haven't used C# much over the last year due to job change but always felt like one of the most mature languages out there. Now working in Go and it's a bit frustrating in comparison.

<comment><USER6> Go isn't designed to feel mature, it's designed to be boring and effective. It's designed to keep code complexity low even as the complexity of problems and solutions increases. It's designed to allow large teams of medium-skill programmers to consistently produce safe and effective solutions. The most precise description ive heard to date is: "Go is a _get shit done_ language."

<comment><USER17> Basically it is designed for writing boilerplate libraries and code generators to cover up lack of language features, which even well known projects are forced to make use of (k8s).

I bet a G2EE variant isn't too far away.

<comment><USER18> Yet in the infrastructure side, it's much more used than c#. Go figure.

<comment><USER17> You again.

What infrastructure, those riding the consulting and conference Docker and K8s 2019 wave fad?!?

<comment><USER18> Right now I'm using prometheus and grafana to monitor around 8k database servers (sql server too BTW). We have pricing applications using influxdb. Docker. Openstack. Minio. We also have mattermost.

All of this in a conservative big bank. My friends in the banking sector tell the same story.

True there are lots of c# enterprisey web apps.

However given the amount of boilerplate you describe, I cannot understand how such useful and reliable tools can be delivered in Go.

A hint:just because a language is not to your liking, it does not mean that it is not useful, performant and reliable.

<comment><USER17> One anecdote doesn't make the IT industry.

<comment><USER18> I'm talking about the whole banking sector in France.

<comment><USER17> And yet I haven't seen any of that on our Fortune 500 French clients, which naturally includes banks, go figure.

<comment><USER18> Are you working in operations and infrastructure? My take is that writing enterprisey applications you are not exposed to those tools. I'm in ops.

<comment><USER17> Not personally, but we do have mixed teams.

AWS, Azure, actual hardware racks, plain old VMs, JEE containers, .NET packages, Ansible, Puppet, Chef, whatever scripting stack, but surely not one line of Go related code.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER19> A specific finding from this research is on the front page:

[https://news.ycombinator.com/item?id=20944403](https://news.ycombinator.com/item?id=20944403)

Rust was found to be slightly slower than C because of bounds checking, which the compiler keeps even in production builds.

<comment><USER20> Except their answer is wrong, because Rust (LLVM rather) does eliminate bounds checks. They're comparing GCC vs LLVM here more than they are comparing C vs Rust. They should have compiled their C code in LLVM. Their implementation is littered with uses of "unsafe" which means its almost impossible for the compiler to eliminate the bounds checks.

<comment><USER21> There's a per-packet bounds check here [1] which probably can't be eliminated by the compiler because it cycles over the array. I imagine that's noticeable.

[1]: [https://github.com/ixy- languages/ixy.rs/blob/master/src/ixgb...](https://github.com/ixy- languages/ixy.rs/blob/master/src/ixgbe.rs#L175)

<comment><USER22> So the bounds check is:

        queue.bufs_in_use[rx_index]

If so the bounds check could possibly be safely eliminated by the programmer because I think `wrap_ring` ensures that rx_index will always be in bounds?

<comment><USER21> Yes. It wouldn't be too unidiomatic to use get_unchecked in those two places, perhaps with a debug_assert! in place.

It would be really nice if this wasn't needed, but it's a valid use of unsafe code.

</comment></comment></comment></comment></comment><comment><USER23> So why the difference in "language" speeds?

You have some the results not quite following the conventional expectation. For example the Swift implementation is as slow as JavaScript. JavaScript is a lot faster than Python. Java is considerable slower than the usually very similar C#.

The implementation is fairly complex; so it is a bit hard to see what is going on. But it must be possible to pin the big performance differences implied by the two graphs to something?

<comment><USER24> Python is interpreted bytecode. This means that for every small instruction on the bytecode there's a round trip to the Python interpreter that has to execute that instruction. This is faster than parsing and interpreting at the same time, such as shells often do, but it's still a lot slower than JIT compilers.

Now, a just-in-time (JIT) compiler transforms the code into machine code at runtime. Usually from bytecode. Java, C# JavaScript all use this model predominantly these days. This takes a bit of work during runtime and you cannot afford too complicated optimizations that a C or C++ compiler would do, but it comes close (and for certain reasons is even better sometimes). So that's the main reason why JavaScript is faster than Python. Theres a Python JIT compiler, PyPy, that might close the gap, though. And for Python in particular there are also other options to improve speed somewhat, one of them involves converting the Python code to C. Not too idiomatic, usually, though.

As for Java and C#, that's a point where it can sometimes show that C# has been designed to be a high-level language that can drop down to low levels if needed. C# has pointers and the ability to control memory layout of your data, if you need it. This turns off a lot of niceties and safeties that the language usually offers (you also need the _unsafe_ keyword, which has that name for a reason), but can improve speed. Newer versions of C# increasingly added other features that allow you to _safely_ write code that performs predictably fast. But even value types and reified generics go a long way of making things faster by default than being required to always use classes and the heap.

Java on the other hand has few of those features where the develop is offered low-level control. It has one major advantage, though, in that its own JIT compiler is a lot more advanced and can do some crazy transformations and optimizations. One might argue that Java needs that much magic because you don't have much control at the language level to make things fast, so as far as performance goes between C# and Java this may be pretty much the tradeoff between complicated language and complicated JIT compiler.

As for which benchmark shows Java being faster than C# depends a bit on how the code was written, but recently .NET has become a lot better as well and popular multi-language benchmarks show C# often faster than Java.

</comment></comment><comment><USER25> Surprisingly good performance for the C# implementation!

<comment><USER26> For me, that was the line that surprised me the most. The .NET VM has had a reputation as being a worse variant of the JVM, but it seems that now the tables have turned.

<comment><USER27> Really? To me it was always a runtime VM done right! The .net CLR is much more stable, leak-proof, and performant in my experience. I have .net services that run on servers for years without ever being restarted.

Given that C# and "Rust" are neck and neck, I'd rather have a nice GC language to work with.

<comment><USER28> Go and C# are pretty much neck and neck until the batch size gets large. Rust is always ahead of C#.

</comment></comment></comment></comment><comment><USER29> That's a very interesting experiment in many levels. Haven't taken the time to look at the paper yet, but I'm curious of how you got your number of pps vs Gb/s in the README :

"full bidirectional load at 20 Gbit/s with 64 byte packets (29.76 Mpps)". sounds like 20Gb/s should be closer to 40Mpps than to 30Mpps. Did you hit CPU limits on the packet generator, or am I missing some packet header overhead ?

Did you try bigger that 64-byte packets ? I'm curious how various runtimes would handle that.

And how long did you run the benchmarks ? I couldn't really figure it out from the github or the paper. Mostly wondering if java and other Gc'd language showed improvement or degradation over time. I could see the JITs kicking in, but I could also see the GCs causing latency spikes.

<comment><USER30> > am I missing some packet header overhead ?

Yes: Ethernet adds 20 bytes: 8 byte preamble/start of frame delimiter + 12 byte interframe gap

=> the "on-the-wire" size is actually 84-bytes

=> 20Gbps/84-bytes = 29.76Mpps

> Did you try bigger that 64-byte packets ? I'm curious how various runtimes > would handle that.

In typical forwarding, packet size does not impact forwaring that much until you hit some bandwidth limit (PCIe, DDR and/or L3 cache) because you only touch the packet header (typically the 1st 64-bytes cacheline in the packet). The data transfer itself will be done by NIC DMA.

<comment><USER4> PCIe bandwidth also decreases with increasing packet size as there's a lot of overhead per packet. Memory isn't used, it's all handled in cache, hitting main memory is super slow.

</comment></comment></comment><comment><USER31> The fact that Go is slower than C# really amazes me! Not long ago I switched from C# to Go on a project for performance reasons, but maybe I need to go back.

<comment><USER32> What made you come to the conclusion that golang was faster than C#? The hype and claims we see in blogs that are not backed up by anything?

Both C# and Java are faster than golang.

<comment><USER33> Usually where I see C# slow down is not because of the language, but because of over engineered "enterprisey" solutions that Java has a bad rep for e.g. having things like a FactoryProviderFactory type idioms.

A lot of the projects I work on, for instance, heavily utilize dependency injection for no gain. There's only one implementation, theres no test mocks. Its just overengineered and obsfuscated for no reason.

Coming from a predominantly C++ background, we eschew virtual wherever possible, favoring compile time polymorphism to runtime whenever possible, because we're cognizant of the overhead of the indirect dispact and likely loss of optimized opportunities to inline trivial calls.

For sure, one can write C# or Java that can keep up, or even outperform C++ in some circumstances, but youre not going to do it with "enterprise" patterns hiding behind interfaces and factories and dependency injection.

<comment><USER17> That isn't how Turbo Vision, OWL, VCL, MFC, Motif++, PowerPlant, C Set++, ATL, Qt, Unreal, COM/UWP, wxWidgets, JUCE look like.

There are the CppCon talks, the Modern C++ advocacy, and then there is the code that everyone at most corporations actually write.

<comment><USER34> Virtual dispatch is particularly well suited for constructing dynamic GUIs at runtime. Doesn't mean that "everyone" is writing code like that.

<comment><USER17> COM/UWP is not only for GUIs, it is the full area of modern Windows APIs.

Then there are ORM like the ill fated POET.

Yeah just like not everyone is writing code that "eschew virtual wherever possible, favoring compile time polymorphism to runtime whenever possible", specially on large corporations with mixed language teams.

Beyond C++ conference talks, I am yet to see stuff like SFINAE and tag dispatching in the C++ code I occasionally deal with. Grated those are libraries that get called from Java/.NET projects.

<comment><USER33> I have written a fair amount of C++ template metaprogramming and policy based libraries. One library I wrote, in particular, was a templated generic matching engine primarily used in the self-clearing of trades. Through template policies, it could be configured to do one-to-one, on-to-many, many- to-many matching based upon template args, for example. I also did a bit of SFINAE in writing a home-grown ORM lib. I haven't really written any libs using tag dispatching, but I've certainly used my fair share (looking at you, Boost MultiIndex).

You don't usually see these sorts of types wrapped for Java or .Net, and if they are, you usually have some sort of proxy in between to hide the templates.

</comment></comment></comment></comment></comment></comment></comment><comment> non-entity Is there a compelling reason to write high level user mode drivers like this over traditional kernel drivers? I remember finding this repo a few years back and being fascinated.

<comment><USER17> Traditional monolithic kernel drivers are on their way out, specially given security concerns.

[https://developer.apple.com/videos/play/wwdc2019/702/](https://developer.apple.com/videos/play/wwdc2019/702/)

[https://source.android.com/devices/architecture/hal- types](https://source.android.com/devices/architecture/hal-types)

[https://docs.microsoft.com/en-us/windows- hardware/drivers/de...](https://docs.microsoft.com/en-us/windows- hardware/drivers/develop/getting-started-with-universal-drivers)

<comment> non-entity Interesting. I suppose the ability to (almost?) debug drivers like I would any other user-mode applications could be a big win.

</comment></comment></comment><comment><USER35> Rust has definitely earned my respect.

Someone add D lang to this test! I want to know!

</comment><comment><USER36> It misses C++

<comment><USER17> While C++ is way better than using C, it doesn't forbid "writing C with C++ compiler", does rendering useless all the safety features it offers, if one isn't allowed to tame the team via static analysis tooling.

</comment></comment><comment><USER37> This is good.

</comment><comment><USER38> I am calling in question validity of this project as a benchmark.

The author asserts, that "it's virtually impossible to write allocation-free idiomatic Java code, so we still allocate... 20 bytes on average per forwarded packet". This sounds questionable, — does that mean, that he actually performs a JVM memory allocation for _every_ packet?! Furthermore, the specifics of memory management look murky. One implementation uses "volatile" C writes [1] (simply storing data to memory). Another implementations of the same thing uses a full CPU memory barrier [2]. Which one is right?

In my opinion, significant inconsistencies between implementations render any comparison between them invalid. And when a whole cross-language test suite is written by one person, you can be sure, that they don't really excel in many of those languages.

This is why I like Benchmark Game — all benchmarks are submitted by users, so they are a lot closer to how a real-world decent programmers can solve the problem. Still not perfect, but at least that counts as an attempt.

1: [https://github.com/ixy- languages/ixy.java/blob/fcad50339e537...](https://github.com/ixy- languages/ixy.java/blob/fcad50339e5372d7cc1cc53d7a7360bec04aaebd/ixy/src/main/java/de/tum/in/net/ixy/ixgbe/IxgbeTxQueue.java)

2: [https://github.com/ixy- languages/ixy.java/blob/fcad50339e537...](https://github.com/ixy- languages/ixy.java/blob/fcad50339e5372d7cc1cc53d7a7360bec04aaebd/ixy/src/ixy/c/ixy.c)

<comment><USER4> Java reaches 52% of C speed in the benchmark game ("fastest measurement at the largest workload" data set, geometric mean), we reach 38%. Seems like our implementation is within a reasonable range for something that's usually not done in Java.

A full memory barrier is not required, but some languages only offer that. For example, go had the same problem. It's not a bottleneck because it goes to MMIO PCIe space which is super slow anyways (awaits a whole PCIe roundtrip).

And no, it obviously wasn't written by only one person but a team of 10.

No, we are not saying that we allocate for every packet. We say that we allocate 20 bytes on average per packet.

<comment><USER39> Java support all sorts of memory barriers via VarHandles (see GET_OPAQUE/SET_OPAQUE). VarHandles can be created from contents of ByteBuffers:

[https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/M...](https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.html#byteBufferViewVarHandle- java.lang.Class-java.nio.ByteOrder-)</comment></comment></comment>


Terrorism is not about terror - <AUTHOR> http://www.gwern.net/Terrorism%20is%20not%20about%20Terror <comment><AUTHOR> Terrorism is about finding an "in" group. How do you combat this?

My favorite quote about how the internet impacts terrorism: "If we see terrorism as more of a tribal or gang activity than political activism or warfare, then online connections become especially important to our analysis, otherwise we will be fooled by so-called lone wolves. Earlier 'lone wolves' like bombers Timothy McVeigh or Eric Robert Rudolph turn out on closer inspection to have ties, social & otherwise, to like-minded people; McVeigh lived with several other extremists and was taught his bomb-making skills by the Nichols, who also built the final bomb with him, while Rudolph remained on the run for several years in a community that wrote songs and sold t-shirts to praise him and was ultimately caught clean-shaven & wearing new sneakers. Lone wolves who genuinely had no contact with their confreres, such as Ted Kaczynski, are vanishingly rare exceptions among the dozens of thousands of terrorist attacks in the 20th century, and as rare exceptions, otherwise implausible explanations like mental disease account for them without trouble."</comment>


Eskil – a GUI front end to diff - <AUTHOR> http://eskil.tcl.tk/ <comment><USER2> I was recently looking for a fast diff gui, one that would:

\- work well with longer files and long lines (i don't know why many of them are very slow on larger files)

\- have line wrap option (quite unusual it seems)

\- have diff ignoring whitespaces (you would be surprised that not all diff guis support this)

\- the best would be to use the same algo that "git diff" does

\- if not then any other that works on character-level.

Of course you may use the vanilla git diff, which is really great (even outside of git repositories), but it is not as handy as GUI sometimes.

The closest GUI was meld, but it's not perfect either... macvim with some diff plugin was also good, but not perfect.

<comment><USER2> Ok, checked this one against my diffs, and here's some observations:

\+ eskill has a good set of diff algorithms, char level, word level, etc.

\- lacks line wrapping

\- a bit sluggish (but it's not that bad)

Would recommend trying.

<comment><USER3> Eskil creator here. Thanks for the + :-) Line wrapping has been on my low priority todo list forever, but since it is so rarely needed in my usage it tends to not happen. Line zoom is enough for the rare circumstances I have long lines. I'll give it some more thought.

</comment></comment></comment><comment><USER4> What are its current or planned advantages to other tools, e.g. Meld?

<comment><USER3> Among its less visible advantages, it has a lot of ways to resolve tricky diffs, which comes in handy sometimes. Like manually aligning lines, and applying search/replace filters. Try the built in tutorial to see some of it.

But the main advantage is that it presents diffs the way I like it, naturally since I wrote it ;-)

<comment><USER5> Fossil diff is the killer feature for me. Thanks for your great work.

</comment></comment></comment><comment><USER6> I've been using kdiff3 for years (it is also my default git mergetool). I see others use Meld. Ediff (emacs' diff). What are other open source/free tools like that people use?

</comment><comment><USER7> I like tkdiff and I am wondering why Eskil is not a series of patch on top of tkdiff instead of a different program. In fact, [http://wiki.tcl.tk/3773](http://wiki.tcl.tk/3773) says:

        See also eskil for an extended version of tkdiff.

        Eskil is an updated version of tkdiff (still pure tcl) that includes:

        ... features ...

        I highly recommend eskil over tkdiff. tjk

I must be lacking some context, so why not have a single program? Is Eskil that program and tkdiff obsolete?

<comment><USER3> Eskil is a completely unrelated to tkdiff. The only relation is that they both are written in Tcl/Tk. Don't know why the wiki says what it says.

</comment></comment><comment><USER8> This tool looks interesting from the screenshots in the website. I wonder if some things that diff doesn't do well are handled separately by this tool.

Or, I do not know how to make diff do it well - for example, show added similar "paragraph" in better context. If I have a for loop and add an identical for loop after current one, diff will typically show the new one inserted in between the first part of the existing for loop and the last line of the existing for loop. The perforce diff client seems to get this right and shows the new for loop inserted after the current one.

Aside from that, what I miss in some GUI diff clients is the way vimdiff works. By default, vimdiff collapses the similar areas and shows only the diff parts noncollapsed. This is very useful when I am looking at a diff of a large file and the diff portions are far apart. It helps get a quick overview of the differences in the entire file.

</comment><comment><USER9> Meld has become flaky in recent months (years?) after the switch to GTK3. Scrolling often gets 'stuck', mercurial support has regressions and performance is much slower on newer hardware than I remember it being on something barebones. I'm definitely looking at Eskill as an alternative. A graphical diff is just one of those tools I can't live without.

</comment><comment><USER10> Not so great on Mac OS X. For remote diffs over SSH I don't mind vimdiff to be honest, but sometimes I just want a graphical diff locally. I miss meld from GNU/Linux and can't bring myself to brew install it because GTK on Mac OS X is ugh.

<comment><USER11> Just to be sure: you're aware of FileMerge/opendiff? It's installed by default. It's not the greatest, but it's there.

<comment><USER10> Wow, I didn't know about opendiff. Indeed, I will use that when I need something graphical. Cheers!

<comment><USER12> Hmm, I don't know what's with meld/etc, but I always prefer MacVim (mvim -d) over FileMerge on mac.

</comment></comment></comment></comment><comment><USER12> Being vimer, I always fail to see the breakthrough in such news.

<comment><USER13> Similarly as an Emacs user it seems that all the shiny graphical tools are no better than ediff.

</comment></comment><comment><USER14> Can it diff remote locations? sftp://user@host/path

<comment><USER3> No, but that sounds like a useful feature to add. Thanks.

</comment></comment><comment><USER15> Looks like the visual diff that came with Microsoft SourceSafe.

</comment><comment><USER16> I'will definitely use it. Tcl/Tk fan.

<comment><USER17> Out of curiosity, why does it matter what language it's written in? How would that in any way affect one's decision to use or not use it?

</comment></comment><comment><USER18> Is it better than vimdiff?

<comment><USER12> vimdiff is just perfect. How can $subj be any better than perfect? That can be only reason why your [effectively unconstructive] comment gets [constructively explained] downvotes.</comment></comment>


Ask HN: Name a company in Canada with a name trademarked in the USA - <AUTHOR>

I found a great name for my company, let's call it "XYZ Labs", and xyzlabs.com is available. However, xyz.com is taken by a company called "XYZ Software" which has the trademark in the US for "XYZ Software" under the description: "computer programs for use in computer networking, business, and education."
That seems pretty broad to me. I develop software which is used in education. But it's a completely different category of application than theirs. Also, I am in Canada, where XYZ is not trademarked at all, except as a beachwear company.
Am I safe to buy xyzlabs.com and do business as "XYZ Labs"? Or is this a bad idea? <comment><USER2> If you plan to do business (collect money from people in the US), then yes, in my opinion it is a bad idea.

</comment><comment><USER3> Look closely at the trademark for "XYZ Software" and its specifications.

Does it specify XYZ alone as being part of the trademark or does it apply only when used together with Software?</comment>


The Future of the Windows Subsystem for Linux - <AUTHOR> https://www.thurrott.com/windows/windows-10/236646/the-future-of-the-windows-subsystem-for-linux <comment><USER2> For the life of me, I just can not see why WSL is necessary.

If you want to run Linux, just run Linux. If you want to run Linux while also using Windows, then run Linux in a Windows Virtualbox. Or perhaps run Windows in a Linux Virtualbox.

But WSL is this weird jumble just like WINE is, that's neither one pure thing nor the other.

<comment><USER3> We're required to use windows for the main dev tools. The Linux eco just was forced upon us with the begin of ruby, npm, node and so on Aas the windows support was just bad. I'm happy to have wsl for the small Linux tasks I have.

</comment></comment><comment><USER4> Tl;dr: You will be able to run Wayland and maybe even X programs. And CUDA and, maybe, Vulkan.</comment>


We are in the final years of our internet - <AUTHOR> http://stupidiswinning.tumblr.com/post/40268049719/we-are-in-the-final-years-of-the-browser-and-its-going <comment><USER2> There is quite a lot of black and white thinking here which is the primary problem.

Look, consumer technology is always a mess - apps vs web vs native vs desktop vs mobile vs ... etc etc. The truth of it is that we have so many choices because technology is meant to solve individual problems as best as possible.

There will never be one winner.

I think the author assumes just because tablets and apps are growing (I assume this is still happening) that this is the end game. Its not - tablets and apps certainly will be prominent for what they are good at. The app environment sucks for some things. Free flow of information and low friction information searching certainly is one use case where the web is so so much better than apps can ever be in their current form.

Lastly "apps" have always existed. Nintendo games were for all intensive purposes apps in that they were self confined experiences you played on what essentially was a technological black box. In truth if you look historically at how we have been using computing - things are more the same than different - you just have to remove the silly titles we give everything.

<comment><USER3> "There will never be one winner."

There will, however, be many losers, and PCs are going to be one of those losers. The form factor won't die, but the philosophy and freedom will die. It is inevitable, given the priorities people have and the money to be made.

If things continue going the way they are going, the only computers that will give users the freedom to do what we can do now will be locked away in research labs, available only to the lucky few who can land jobs in such places. These computers will be so expensive that only people who are doing funded research will be able to afford them. The next best thing will be computers targeted at developers (perhaps "debug" computers), which will be too expensive for most people to consider buying them but (hopefully) inexpensive enough for hackers to get their hands on them; these will still be loaded with restrictions, but at least the user will be able to write and run code and use a debugger. The only computers that will be anywhere near the price point that most people will consider paying will be those that are restricted to running software that was signed by some large corporation (i.e. only approved programs), with only the macro systems for a few "professional" programs (priced beyond what most people can afford) being available for user programming.

Why would software or hardware companies want it any other way? The next "big thing" will have to be filtered through the app store system, and the little startup that makes that software will be bought out before they can overtake the established players. The big media companies will form partnerships with the companies that control app stores (or just mergers, so that the whole stack is controlled by the same entity) that will be immensely profitable. ESPN will require all of Dell's customers to buy the ESPN app, without the option to get a refund or to remove it, and any company that does not take that deal will be barred from having the ESPN app on any of their products. Governments will love it -- no more uncontrolled cryptography, no more Wikileaks, CALEA-style laws for computers, and a chance to spot political movements before they take hold.

"Free flow of information and low friction information searching certainly is one use case where the web is so so much better than apps can ever be in their current form"

You can just pay $1/mo. for a search engine app, right? It will have an integrated web browser/hook into the installed browser. Not much different from having a Bloomberg terminal on your desk, except that it would be terrible for consumers (but profitable for businesses).

"Lastly "apps" have always existed. Nintendo games..."

Gary McGraw has pushed the idea that the security systems seen in video games ultimately become the security systems on all consumer computing devices. At one time, this app store model was something only video game consoles had; now we see it on tablets, and soon we will see it in other form factors. Video game consoles have highly expensive, hard-to-buy "debug" versions and developer systems; is there any reason to think that such a system could not come to exist for other consumer electronics?

<comment><USER4> I actually don't think that computers capable of doing stuff other than media consumption will get more expensive. I actually think the opposite will happen and you can already see the start of this.

Look at things like Raspberry Pi , ODROID and arduino. Very hackable stuff at rock bottom prices. With faster internet connectivity comes access to remote virtual machines which can be rented by the hour and make stuff like huge scale data processing accessible to the teenage hacker in his bedroom.

Ultimately the future will require more people to understand not just how to use the tech on a basic level but people who can innovate with it.

If the US doesn't do this , another nation will and be all the richer for it.

Of course there will always be a market for passive consumption type devices as well as specialised ones. It's just that these used to be radios , TVs , CD players , calculators and nintendo consoles now they are iPads.

<comment><USER5> _"Ultimately the future will require more people to understand not just how to use the tech on a basic level but people who can innovate with it."_

This. While in one sense it is profitable to rope off technology and control it - having the general population embrace the skill set of working with computers is far more profitable overall for everyone.

In 100 years, every single person will have grown up in a world dominated by electronics with the presence of the internet. Being a fact of life makes the barrier of entry a lot lower than it is today when there are more options.

</comment></comment></comment></comment><comment><USER6> The #1 most-used app on the iPad is... Safari. [[http://rjionline.org/news/rji-dpa-spring-2011-ipad-survey- re...](http://rjionline.org/news/rji-dpa-spring-2011-ipad-survey-results)] And given the nature of the survey ("Open question without prompts") I expect the reported figure (21%) is much too low, because users don't even know what a browser is, half the time. It wouldn't exactly be a shock if half of the respondents who mentioned the New York Times app were really reading the Times through Safari.

The bulk of the non-browser apps mentioned are ways to read things (NYT, WSJ, USA Today, AP News, WashPo, Kindle app).

That is, the bulk of the non-browser apps mentioned are more-or-less tarted-up browsers. So, for that matter, are the likes of Siri and Google Search.

I don't think it's the browser that's in danger here.

</comment><comment><USER7> >Whatsapp has won messaging, text and instant. What will Twitter and Skype do?

Seriously? Operating a centralized walled garden does not satisfy any definition of _won messaging_ when you consider the architecture of the internet.

Nobody has won instant messaging until they all interoperate on the same capacity that email is interoperable, defined by an RFC.

<comment><USER8> A standard defined by an RFC will be a win for consumers, not sure if it'll be a win for companies though. If whatsapp can create a monopoly and make money from it, that's a win for them.

<comment><USER7> We should be focused on wins for technology. This site is after all called _Hacker_ News, not startup of the week entrepreneur news

</comment></comment></comment><comment><USER9> Punditry at its worst. I would have called it clueless, but the truth is that nobody knows, and given evidence, this does not even seem like a plausible prediction.

Sure you can say "mobiles apps grew so and so". But then I can say that given the growth of a newborn child, 60 years old adult should be 50m tall.

<comment><USER10> Agreed. I clicked the article thinking it would be about how regulation is threatening the internet. After 2 sentences I saw it was another article that the old web is dead and that we will only use apps in the future... so tired of hearing that line. As many others have stated, walled garden vs. free- flowing information, etc. Drives me nuts.

</comment></comment><comment><USER4> This is somewhat all over the place without a lot of supporting evidence.

Is the author suggesting that the web will die, and be replaced by thousands of special purpose apps that must be downloaded from the app store?

<comment><USER11> Peeling back the exterior and getting at the meat of the argument, I think the author is really arguing that the future "makers" of things are having their perception of the internet and computing shaped by their interactions with walled garden devices and controlled environments. Building off of this, they will proceed to continue perpetuating a world emphasized by unique apps and walled gardens instead of the creativity of current and prior hacker generations. What would be really interesting is if the author did a study looking at past generations and how their initial perception of connectivity shaped what they built using it. I.E. all the phone phreakers and hardware hackers in the 70s and 80s laying the foundation for modern computing.

<comment><USER12> A more concise argument is that your Android tablet that is Junior's first computing experience is significantly less hackable than your Commodore 64, which was more hackable than your DOS machine, which was more hackable than your Windows 95 machine, which was more hackable than your XP machine, et. cetera.

Devices provide higher levels of abstraction from the tweak and tunables on the system, but only now are we seeing devices where there is no easy way to actually write code for them, on them. You can't write an iphone app on an actual iphone, and I don't know if you can compile an apk on Android (I know Java ides on Android exist, though).

About the only thing you can do is write web pages you can open locally. That still works. And I don't see walled apps overtaking the expanse of the internet - there is just too much content and experience only found surfing old crummy html documents you won't get on your walled garden facebook app experience. And Android is taking the smartphone market, while being relatively open for a platform with the rooting and loading of any apk you want (if you set the option).

<comment><USER13> Really good (long) summary, zanny. That was what I had garnered from the article's musings. It's something I've thought about a lot myself actually. How would I give a child the opportunity to learn to hack or program if they wanted to? For me it started with c-64 basic cause it wasn't compiled, then visual basic when I was like 13... I guess C# or Java could probably be taught very easily, but it feels like starting out learning martial arts by teaching them advanced maneuvers. It feels like the wax on, wax off approach really is the wise course but showing them QBasic or something hardly seems like a step forward.

<comment><USER12> I'm sure if they have the right interest, they will investigate how these devices work. It isn't as nice as being forced into learning a lot of it by necessity of the tech (think TTY only devices) but I'm sure most who possess the correct spark might end up writing web apps instead of bash scripts.

</comment></comment></comment></comment></comment><comment><USER14> Am I the only one who stop reading it as soon as I noticed that the author confuses the Web with the Internet?

</comment><comment><USER5> On that blog, stupid is definitely winning.

<comment><USER15> The insight and creativity with which you discuss the concepts brought up in the article greatly raises the HN level of discourse.

<comment><USER5> Aw come on zero, losen your collar a little bit. I've conveyed my well thought out post with brevity, pregnant with purpose.

</comment></comment></comment><comment><USER16> This was an interesting read that raises many questions, but much of the analysis is ultimately flawed because the author confuses many concepts.

Internet is not a human-computer interface. It is true that most native apps, non-browser apps on mobile and tablets use touch as the main point of interaction. And it is true that many people browse web via http or https using a keyboard and a mouse. But who knows how this will change in the future? People change. Web standards converges and diverges constantly. Technology evolves. Browsers today already do unthinkable things. There's no reason to completely assume that the primary HCI of web will remain as keyboard and mouse. Nor should we completely assume that the touch interface on tablets are the be it and end all method of interacting with native apps.

</comment><comment><USER17> Every generation must discover, and embellish in its own way, panics about the "Imminent Death of the Net".

</comment><comment><USER18> "How will our children learn to create digitally?" THAT is the problem, tablets and smartphones are designed to CONSUME, not create: There are some crazy guys who write code in their iPad, but let's face it, 95% of users only consume content in their devices.

</comment><comment><USER19> The only real insight I picked out of this was that children today are living in a walled garden. Before, we had to know about drivers, installers, executables, the filesystem, etc. Today, a lot of that is hidden. A benefit of that is that the system is a lot more stable and people are allowed to maintain the abstraction provided by the os. A question that Im curious about in the future is whether the success of the abstraction will prevent kids from peeking behind the curtain for simple lack of need. Lastly, will that in some way affect the technological capabilities of the kids?

</comment><comment><USER20> I have an iPad and love it, but haven't bought a single app for it. Sure I use some free apps, bit what is the single most used app? Safari, for, wait for it, browsing the web.

It will be interesting to see how today's kids get into content generation with less keyboard experience and I do believe that tablets will rose over huge chunks of the internet user experience--it is just too easy to consume content using one.

But the end the open web of google and web search? And the rise of the special purpose app for everything? I don't think so. Not as long as tablets ship with standards compliant browsers.

</comment><comment><USER21> I was looking through old computer magazines from the 1980s and thinking about my youth when there were numerous incompatible microcomputer platforms that had some popularity at some point such as

* TRS-80 Models I and III * VIC 20, C-64, C-128 * Apple II, II+, IIe, ... * CP/M systems based on the S-100 bus * IBM PC

and way too many others to mention. Eventually the PC and the Mac won out, except for a few Amiga fanatics and a specialized "workstation" marked served by vendors like Sun.

It seems the current state of things isn't nearly as fragmented as that!

<comment><USER22> You had the MSX computers, but they weren't so popular. At least, not in the UK.

</comment></comment><comment><USER23> Any kid whose parents are hackers, writers, journalists, scientists, accountants or any other profession that actually has to type stuff on their keyboards will still have real computers in their homes in addition to tablets.

For that matter, kids might still find machines with big displays and keyboards useful, since writing an essay or term paper on an iPad probably wouldn't be a very pleasant experience. (Or do kids not get writing assignments in school anymore?)

</comment><comment><USER24> Stupid is not winning (apart from alttab's keen observation).

</comment><comment><USER25> Regarding the gaming angle, I think there will always be some kind of premium gaming platform for people who aren't satisfied with phone and tablet games. It may well be that the standard gamer demographic continues to shift to an older crowd as we give our kids tablets instead of Wii's or Playstations and they have to wait until they're older to get access to a real console or high- end PC.

</comment><comment><USER26> You voted with your startup creating apps instead of websites and your wallet by buying apps that this is what you wanted. That I warned you (<http://drupal4hu.com/future/freedom>) doesn't matter but Douglas Rushkoff warned you and you still didn't listen. So crying over the lost freedom is all that's left, congratulations.

</comment><comment><USER27> The early web was scattered because there was infrastructure in place but the initiatives were short-lived, diverged, or motivated by personal reasons.

It's the nature of our market - or maybe our culture - for things to agglomerate and monopolize though. Just notice how it's more likely you're wearing shoes made by a multinational company, not the shoemaker next door, or even yourself.

</comment><comment><USER28> I have kids, and I disagree that things are going to play out as black-and- white as the author suggests. My daughter (2) does stick to the curated appstore/app world on her tablet, but she's just as anxious to get her hands on my laptop and uses it to do different things. Like typing "cat" into google images to see cat pictures. Just like the rest of us ;)

</comment><comment><USER29> Tablets and similar devices are great at what they do, but I can't think of one use case (aside from extreme mobility) where they exceed desktop and laptop computers.

</comment><comment><USER30> I agree, we are in the final 60 years of our internet

</comment><comment><USER31> Bad enough to flag.

</comment><comment><USER32> when will URLs die?

<comment><USER33> It seems like you are implying that you want them to die. Can you elaborate on this? I think URIs and DNS are pretty nice if used correctly. How else would you identify things across protocols and the entire web?</comment></comment>


Show HN: Product Kitty, a Product Hunt Browser Built in React Native - <AUTHOR> https://github.com/rkho/product-hunt-react-native

<comment><AUTHOR> Built this really fun React Native application recently. It lets you:

\+ View the latest featured products on Product Hunt (and goes back to previous days as you scroll down)

\+ View the latest 50 featured collections as well as collections that are created by users

\+ View similar products to the one you're currently viewing

\+ View user profiles, including their list of upvoted products, submissions, collections, and any products they've been tagged as makers of.

Hoping to get write access to the API so I can allow users to comment and post through the application.

It's also live on the App Store: [https://itunes.apple.com/us/app/product- kitty/id1018558174?l...](https://itunes.apple.com/us/app/product- kitty/id1018558174?ls=1&mt=8) (it may take a few minutes to actually show up):

</comment><comment><USER2> Looks great! Can't wait to try it out!</comment>


Ask HN: Travis CI having troubles right now? - <AUTHOR> It seems doing the AWS mistake showing all-green status for last 3 hours.
Logs are not available 3 hours already. Even jobs info became unavailable now.
Their status page is still green, and their Twitter is silent. What is going on? <comment><USER2> I'm having intermittent issues viewing build logs on Travis right now. Lots of people complaining about the same thing on Twitter.

</comment><comment><USER3> They definitely had an outage related to S3 -- both of my projects got better an hour or two ago.

<comment><AUTHOR> Yeah, seen it, but then it became green and not working.

Now the status page has been updated to yellow [https://www.traviscistatus.com/incidents/hx7cnxbch9xf](https://www.traviscistatus.com/incidents/hx7cnxbch9xf)</comment></comment>


Bokeh – a Python interactive visualization library - <AUTHOR> http://bokeh.pydata.org/en/latest/ <comment><USER3> Since I have never really liked Matplotlib I would really like to learn Bokeh, but unfortunately its inability to export its visualizations in SVG or similar formats makes it kind of useless for me as a scientist wanting to publish my results.

<comment><USER4> If you're not wedded to Python, have a gander at Gnuplot. Despite its quirks, it's my favorite plotting tool.

<comment><USER5> I severely dislike Gnuplot, but then again, I am wedded to Python. I'm not sure how versatile it is, at least from the slightly acquainted position I am.

<comment><USER6> FYI it's really dead simple to write new "bokeh" command line tools as Bokeh apps now. We already have several ("bokeh html" and "bokeh json") but we are definitely interested in make a gnuplot-like bokeh command line tool that you can just point at a CSV or log file and get a visualization right out. E.g. something like "bokeh graph foo.log -x time -y connections --tail" would create an automatically updating visualization of a growing log file. Or if anyone is interested in this functionality and wants to help speed along the development, please let us know!

</comment></comment></comment></comment><comment><USER7> "Bokeh is a Python interactive visualization library that targets modern web browsers for presentation."

A description of how one gets from Python to a web browser display would be nice. Is there a translation from Python to Javascript somewhere? Is there a Python web server backend? Are there dynamic visual updates or does this thing just generate a static output like a .png file?

<comment><USER8> my question exactly. What does this do that I can't do with highcharts?

<comment><USER9> I forgot the really important difference between bokeh and highcharts - <USER2> can throw 100k points at it and have it be fully interactive without your browser blowing up!

<comment><USER10> HiCharts has dynamically loadable data. I have a database that has 120M points. I just keep some pre aggregated data to populate the wide periods. As you zoom it it selects smaller aggregation sets. (Think; 1 month for 10 years, 2 weeks for 1 year, etc). If you have the data you only need about 20 lines of JS so support this with json. That said, bokeh requires little to no javascript and that's a huge advantage if you need to get multiple visualizations. It does, pretty much stock, what you can do with HiCharts without needing as much full-stack expertise.

</comment></comment></comment></comment><comment><USER11> Great library, specially useful for doing streaming visualizations. I gave a talk last year at PyData Singapore about it, slides are available here - [http://asankhaya.github.io/professional.html#PyDataSG](http://asankhaya.github.io/professional.html#PyDataSG)

<comment><USER12> That's super cool.

</comment></comment><comment><USER13> Strange name, it refers to the way a camera lens blurs out-of-focus objects (especially highlights) into a disc shape [1]. I was expecting an image processing library.

Bokeh is widely discussed by photography equipment aficionados as one of the main distinguishing traits of different lens designs besides sharpness, distortion, etc.

edit: maybe it is an allusion to bubble plots?

[1] [https://www.google.com/search?q=bokeh&tbm=isch](https://www.google.com/search?q=bokeh&tbm=isch)

<comment><USER9> i believe the idea behind the name is that bokeh (python) is to data viz what bokeh is to photography - its a tool that brings the important data into focus.....

<comment><USER14> bokeh by definition is talking about the out of focus stuff, which is arguably the less important part of a shot.

<comment><USER15> Which brings in focus elements into sharper attention.

<comment><USER16> And a photo that is entirely bokeh?

[http://i.imgur.com/siLB9C3.jpg](http://i.imgur.com/siLB9C3.jpg)

</comment></comment></comment></comment></comment><comment><USER17> I've been playing around with this library and really like it. Python really is that much easier to use than R isn't it? Too bad most on the libraries are still in R. It's strange that there isn't a language written on top of R that is less troublesome to use or learn.

<comment><USER18> Call the libraries with Rpy2, or build models using a DAG of functions and distributions with pymc 3

<comment><USER17> Thanks i'll check that out.

</comment></comment></comment><comment><USER19> Ruby is so behind in this... :/ The closest thing I could find, which is only just a proposed enhancement in a github project, is considering adding ploty.js to Flammarion (a new Ruby GUI toolkit using the browser as working window), or maybe Ruby Processing to some extent?

<comment><USER20> Actually, we'd LOVE to get a bokeh.rb project going! Most of the power of Bokeh is in the BokehJS runtime, and in our M-VM-V-C architecture for visualization. The Python code is just generating a JSON representation of the plot, but it's relatively straightforward to do that from any language. You can see front-end implementations in other languages:

* R: [https://github.com/bokeh/rbokeh](https://github.com/bokeh/rbokeh) * Scala: [https://github.com/bokeh/bokeh-scala](https://github.com/bokeh/bokeh-scala) * Julia: [https://github.com/bokeh/Bokeh.jl](https://github.com/bokeh/Bokeh.jl)

The Facebook data science folks even implemented a small Bokeh wrapper in Lua, as part of their iTorch package: [https://github.com/facebook/iTorch/blob/master/Plot.lua](https://github.com/facebook/iTorch/blob/master/Plot.lua)

So, you can see that you can get started with a basic wrapper, and then build up from there. The Lua wrapper is only ~850 lines. If you want to pop over to the mailing list or on our Github, we'd love to help you out.

</comment></comment><comment><USER21> Any chance of getting access to the old tutorials that consisted of scripts with the boilerplate written, but missing the key functionality? I was really enjoying working through them and digging into the docs, then all of a sudden they were taken offline and I was directed to the notebooks.

</comment><comment><USER22> Bokeh is cool, but I've found that it's method for customizing appearance of graphs is a little unintuitive and clunky.

For example, have a look at this cursory analysis of my Reddit comments that I did late last year. Getting the graphs to look nice felt like pulling teeth at the time.

[http://nbviewer.jupyter.org/github/Niksko/redditCommentData/...](http://nbviewer.jupyter.org/github/Niksko/redditCommentData/blob/master/CommentAnalysis.ipynb)

That being said, I'm reasonably happy with the results. And that pannable, zoomable line graph is pretty fancy and is easy to set up.

</comment><comment><USER23> _" To use the Bokeh server with python 2.7, you also need to install Futures package."_

Does this statement mean it supports Py3 and if you want to use it with 2.7 you need Futures? Or is it only Py2.X? [0]

[0] [http://bokeh.pydata.org/en/latest/docs/installation.html](http://bokeh.pydata.org/en/latest/docs/installation.html)

<comment><USER24> Bokeh can be used with both Python 2.7 and Python 3. Additional dependencies are needed to work with Python 2.7. In fact, the "write call-back functions in Python" capability uses a Python to JS compiler called Flexx -- [http://flexx.readthedocs.org/en/latest/flexjs](http://flexx.readthedocs.org/en/latest/flexjs). Flexx is currently Python 3 only (though 2.7 support is coming for it).

<comment><USER23> thx @travisoliphant the requirements are not clear in the docs. This explanation makes more sense.

<comment><USER25> Thanks for the feedback. We keep a continuous and frequent effort to make documentation more efficient and straightforward .. Building clear is a constant/endless effort and feedback is very important.

To clarify, Futures is required for bokeh server as an extra dependency on Python 2.7 since it's not a battery included like for Python > 3.2.. if you conda install Bokeh it should be installed already for you. You can find more info about the bokeh server features here: [http://bokeh.pydata.org/en/latest/docs/user_guide/server.htm...](http://bokeh.pydata.org/en/latest/docs/user_guide/server.html) and here: [http://bokeh.pydata.org/en/latest/docs/user_guide/cli.html#m...](http://bokeh.pydata.org/en/latest/docs/user_guide/cli.html#module- bokeh.command.subcommands.serve)

Flexx, on other hand, is not required by bokeh server itself but instead to define python functions to handle interactivity that runs on the browser (no need of bokeh server). Flexx is used to convert python to JS. You can find more info here: [http://bokeh.pydata.org/en/latest/docs/user_guide/interactio...](http://bokeh.pydata.org/en/latest/docs/user_guide/interaction.html#customjs- with-a-python-function)

<comment><USER23> thx @fpilger, fantastic to see another Py3 ready application. It's clear now. One way to avoid this is either give a install write up an installation guide [0] that explicitly mentions the python version and dependencies.

This is one of the first questions I ask, "is this code Python3 ready?", so I look at the install requirements first. Maybe I'm an edge case. A short cut might be just to say in big words, Python3 ready.

[0] For example this PIL fork, Pillow (python3) [https://pillow.readthedocs.org/en/latest/installation.html](https://pillow.readthedocs.org/en/latest/installation.html)

<comment><USER26> If wanting to know up front if this is designed for the future or the past is an edge case, it's a pretty crowded edge. Python 3 was released eight years ago, and I will ALWAYS choose a competing technology (if it means a language other than Python, that's fine with me) before I'll let Python 2 play any role in a new project I architect.

The first thing I looked for on the Bokeh website was some sort of clear statement on the front page about it being Python 3. I don't have an immediate need for it, but if it's Python 3, I'll keep it in mind. If it's just that euphemistic "Python", it may as well be Cobol for all I'd care.

I thought I'd skim over these comments just in case someone asked the obvious question, and someone did. (Thank you.) And, yes, it is apparently Python 3 enough that we wouldn't need to use any Python 2 if we adopted it.

Very nice. Now I wish I DID need it, but maybe I'll do a side project of my own with it, just for fun. I'll be keeping it in mind in any case. I'll bet "Python 3" is mentioned somewhere on the site, but I didn't spot it right away. It might be worth making it a bit more prominent, since it is an important feature for those designing for the future.

<comment><USER23> _" If wanting to know up front if this is designed for the future or the past is an edge case, it's a pretty crowded edge. Python 3 was released eight years ago"_

good point: a quick search reveals that Py2.x is still in a lot of legacy code (dependencies -- the big one), so I always check first:

\- [https://python3wos.appspot.com/](https://python3wos.appspot.com/)

\- [http://stackoverflow.com/questions/30751668/python-2-vs- pyth...](http://stackoverflow.com/questions/30751668/python-2-vs- python-3-2015-usage)

\- [http://hiltmon.com/blog/2014/01/04/python-its-a- trap/](http://hiltmon.com/blog/2014/01/04/python-its-a-trap/)

\- [https://blog.newrelic.com/2014/01/21/python-3-adoption- web-a...](https://blog.newrelic.com/2014/01/21/python-3-adoption-web-apps/)

\- [http://www.randalolson.com/2015/01/30/python-usage- survey-20...](http://www.randalolson.com/2015/01/30/python-usage-survey-2014/)</comment></comment></comment></comment></comment></comment></comment>


MUI – Lightweight CSS framework that follows Google's Material Design guidelines - <AUTHOR> https://www.muicss.com/ <comment><USER2> It seems from their github profile active development is going on in MUI CSS since Feb, 2015. This CSS framework looks decent. They given some pre-defined layout to start with. Also, first time I seen they given some example for HTML Email template.

They supports React, Angular JS and Web Components. Website is very neat and clean, it's a real pleasure to navigate their site.

I created a simple and short video overview (no installation, or tutorials) [https://www.youtube.com/watch?v=8I2YS7OCkpQ](https://www.youtube.com/watch?v=8I2YS7OCkpQ)

</comment><comment><USER3> I remember the MUI Amiga UI framework, the first that made AmigaOS really look nice.

<comment><USER4> Whenever I see the letters "MUI" this is exactly what I think of!

<comment><USER5> I'm the context of Material Deaign, MUI is short for Material UI, a React based implementation of Material Design. Not sure why the developer of this library would choose the same name, although I can guess. In any case it can only lead to confusion.

</comment></comment></comment><comment><USER6> It was enough to see the home page for a second to know they don't follow Material Design.

The "Download" and "View on GitHub" buttons are 36px high, which violate the minimum 48px touch target height.

This is the kind of thing that really kills the illusion of a universal spec and turns web{sites,apps} into "lookalikes" rather than "the real thing".

</comment><comment><USER7> I've used this. It's easy to use, looks good, and is about a million times easier to set up than Polymer.

</comment><comment><USER8> I wish Apple would have a similar guideline to front-end developers. Here's the closest I can find [https://developer.apple.com/ios/human-interface- guidelines/](https://developer.apple.com/ios/human-interface-guidelines/)

</comment><comment><USER9> This is a beautifully put together website to showcase the library. Simple, fast, functional. All examples and questions covered. Design files are a bonus.

Also, as someone who browses with a content-blocker, a complete rarity to come across a site that works fully with zero 3rd-party host dependencies.

</comment><comment><USER10> As an alternative I've been using [http://materializecss.com](http://materializecss.com) for a while and really love it.

</comment><comment><USER11> Is there a terse bird's-eye view explanation what Material Design is from a programmer's perspective? I tried approaching Google's documentation more than once, but, invariably, got completely overwhelmed every time.

<comment><USER12> It is a design language from Google. It takes a lot of inspiration from flat design but uses elevation (for example with elevated buttons). the idea is that our minds are used to '3D scan' what is in front of us and that it helps identify clickable elements. Color is also used a lot. In an app, the same accent color is used in order to denote clickable elements.

It gives us a framework with mostly good choices (nothing is perfect) in order to build apps (or websites) that users can use more easily since they will gradually get used to its conventions.

</comment></comment><comment><USER13> Hi Everyone - This is Andres, the lead author of MUI. It's exciting to see the project on HN and I'm glad to see some folks are finding it useful. Please let me know if you have any questions or suggestions!</comment>


Ask HN: How do you find critical thinkers/beta testers for your SAAS/Product? - <AUTHOR> tl;dr looking for a way to find critical thinkers/beta testers for my SAAS.
Hi everyone!
The last few weeks I have been developing my own Software as a Service product* hoping to launch soon, however now that I'm coming closer to the first beta release of the software I have no idea what to do next. Obviously marketing should be the first step, if the product is ready to be released. But its not. Instead of that I'm looking for a way to find beta tester and/or people that want to do some critical thinking of the product. Free or paid doesn't matter much, since after all the product should be profitable.
Personally I don't think ProductHunt is the way to go since the product isn't finished. Neither is Hacker News because it would quickly turn into a 'rate my saas' spam I'm afraid. Reddit may be an option. but as far as I know there isn't a sub reddit dedicated to this. The final resort would be actual advertising, but there is no guarantee people would give their feedback.
* I'll put this at the bottom since I don't want to disguise this post as an advertisement, I genuinely want the best way to find other people. But here it is: https://servitor.pw Once again it is rough on the edges and still requires a lot of work.
Edit: I'd like to apologize for any grammer or spelling mistakes, English isn't my native language. <comment><USER2> Reach out to your network/friends, find someone in your target market that can be an early adopter.

If you don't have anyone in your close network you can start reaching out by email to distant connections you might have or try to connect with someone who writes articles, has a podcast that you are a fan of that would be a good fit.

If you approach people the right way it's easy to get connected with 'internet famous' developers.

What stack/framework are you using? Get involved in that community right away. Write articles, go to meet ups. Contribute to the framework. Become part of the community. You'll be able to meet potential early adopters and get to know the community where lots of potential future signups could come from.

Have your early adopters use it and report back with feedback/suggestions.

You can also setup monitoring for when they login, when they get alerts to see how much they are using it, how much value it is providing for them.

Good luck with your SaaS.

</comment><comment><USER3> Who are your target customers? How many of them have you contacted? Whatever the answer, find more of them! And show them your product and ask them to pay you some money in advance to get a lifetime discount. If none bite, find out why.

</comment><comment><USER4> Betalist.com, betapage.co, and getworm.com

Betalist takes time approx. 2-3 months, so you should submit your product right now so that when you are done polishing it more, your product will be live there.

And, if you get enough upvotes on betalist, you will have good amount of sign ups for your platform.

After that you need to engage with them to get the maximum feedback for your platform.

</comment><comment> andrew-lucker Post your product anywhere with a comments section. People love to complain and you will get more (negative) feedback than you can handle.</comment>


Show HN: Add hidden messages to your tweets, Facebook posts, WORD docs and more - <AUTHOR> Hey Hacker News!  This message has a hidden message inside it called a textmark.  Find out what it is at textmark.io, use the find textmark tool.  We created textmark to allow you to protect your valuable content or send hidden messages to friends, the skies the limit.  Just copy and paste this message into the "find textmark" box. <comment><USER2> Interesting. Did some quick poking around.

Looks like the system adds a number of non-printing unicode characters into the string. In this thread title, for example, the textmark is stored in the phrase "message has a hidden message inside." (Thanks, textdiff!)

Specifically, it adds non-printing unicode characters repeatedly. I'd guess these form a unique series, which is mapped to a record in the textmark database.

If I wanted to automatically remove the textmark from a text, in order to steal your content, I would write a script to automatically remove any non- printing unicode characters that appear in it. This StackOverflow answer includes a script achieving precisely that task: [https://stackoverflow.com/a/11598864/6226586](https://stackoverflow.com/a/11598864/6226586)

<comment><USER3> I was thinking the same things like you. Looks like they don't event hide the actual message. They basically hide some sort of ID that points a row in the database. After some research I've made a web-app which can hide any message into any text.

If you wanna look it's on GitHub. Here are the links.

Demo: [https://ozgrozer.github.io/titus/](https://ozgrozer.github.io/titus/)

Source: [https://github.com/ozgrozer/titus/](https://github.com/ozgrozer/titus/)

</comment></comment><comment><USER4> Hi there, great idea. Sort of like text stenography.

Is the message really hidden in this message? Or are you just doing the lookup your database of hidden messages based on the hash of the public message?

Regarding the website UX - after I pasted the message to your text field I thought the hidden message is the youtube video. But that didn't make sense to me and only later I understood the message is above the video.

If you are interested in more UX feedback, feel free to visit [https://usability.testing.exchange](https://usability.testing.exchange) which is a free community platform built exactly for this purpose.

Also, looking at the payment options - I'm missing a pay-per-message approach. For your consideration.

<comment><AUTHOR> Great feedback the message has been updated.

</comment></comment><comment><USER5> Cool idea. Needs a demo to show how it actually works and not just for decoding messages. Like what's the use case.

Also, all those trademark symbols makes this look comically unprofessional.

<comment><AUTHOR> Hey thanks for the feedback! Definitely will add a video shortly just putting it out there to get great feedback like yours.

</comment></comment><comment><USER6> I only changed a word from it and it wouldn't find the trademark anymore. Not that useful IMHO.

</comment><comment><USER7> Hey textmark.io! Steganography is so cool, but ensure ownership without a robust crypto system is a little bit pretentious ... And in some cases, dangerous!</comment>


Ask YC: Single founder company with prototype, how do I improve my odds of being accepted. - <AUTHOR>

I am getting ready to submit my application to YC and I am interested in the advice of others.  I already have a working prototype but the people who were going to develop the company with me aren't ready to commit 100%.  In your opinion, what can I do to improve my odds?<p>thanks <comment><USER2> Single founders get accepted all the time.

IMO, it's a pretty simple formula.

1) Build something people want-- ideally, prove it. Have users. Show that they log in every day and would sooner gnaw off their own arm than lose your app. There are other ways to prove market demand other than loyal users. Find them. Leave no question that people desperately want what you're building.

2) Prove that you can build stuff. Have an impressive and interesting resume for getting stuff launched/shipped-- ideally, you can show that you've build/shipped stuff under great adversity. Show determination.

3) Show that it's a big opportunity. I remember there was a team talking about building a tool to manage D&D campaigns online. That's not a big opportunity. YC only makes "keep the lights on" money with the 10-20m exits. You should hopefully be playing in a space where there is a ghosts chance of a 9-figure exit. They're pretty lenient on this one, I think-- ya never know what can get big.

(IMO) The biggest risk YC is taking on is that no one will want what you're building. The second biggest risk is that you aren't capable enough or determined enough to get it built. Your adding the risk that you can't find a decent co-founder-- which doesn't help.

<comment><USER3> Out of all the ycombinator apps so far, which would you say satisfy number 1? The only one that falls into that category for me, amusingly, is this site.

<comment><USER4> Even youtube doesn't satisfy No. 1 in the early years when there weren't much interesting videos in their database.

<comment><USER3> Yes, but hotmail, gmail, gdocs, skype, firefox, delicious, flickr et al did from word go. Once you started, you never stopped. By all accounts youtube was lucky, got momentum at the right time and struck it rich. It could have been many of the other similar sites in the same position.

I'm not saying its easy to develop something a user would jump off a cliff for, more the contrary. People say it like it's easy, iv'e been doing my own startups for 5+ years now and still don't have one, but i'm working on it just like everyone else here I suppose!

</comment></comment></comment></comment><comment><USER5> It's not the end of the world if your cofounders aren't ready to work full- time. It depends why. If they're in grad school, for example, and only willing to work on the startup on the side, that could be ok, so long as one of you will work full-time. (That's how it was with Viaweb.)

It would not be so good if they merely had jobs they weren't willing to quit to work for the new co. Then it's kind of a vote of no confidence in the new co. And since they'd _be_ the new co, that means a vote of no confidence in themselves. If that's the situation, I'd try to find at least one new cofounder who was more enthusiastic.

</comment><comment><USER6> make a (short -- 3-4 min) screencast. pg et al are probably not going to have time to play with your app to the extent that you'd like (or perhaps at all), plus you get to show your idea/prototype in the best possible light.

if you're feeling lucky, submitting it to news.yc can get some additional attention.

i was a single founder, did this and it helped me 1) bring on a great cofounder and 2) get accepted into yc.

<comment><USER4> i recommend using jing video for the screencast.

</comment></comment><comment><USER7> I applied as a single founder last time, and nobody checked the working prototype I linked to. So it appears that you shouldn't spend your time working on that.

Edit: Or maybe I just didn't make the first cut as webwright suggests below.

<comment><AUTHOR> If you don't mind, what did you do? Did you go after other sources of funding? What are you doing with your prototype now?

<comment><USER8> What's the deal with this 'funding' talk? Why can't you let your customers 'fund' your product?

</comment></comment></comment><comment><USER9> Interesting to see the comments. We are in the same boat as well. We are a team of three people but two are on H1-B visas and cannot quit their jobs (otherwise they'll have to leave within 2 weeks). Anyone in a similar boat?

</comment><comment><USER10> Find a co-founder. If you think you can successfully run the company alone, then don't worry about it. Otherwise, you will need to find more people at some point, so you may as well address that now.

<comment><USER11> If you think you can do it alone you vastly underestimate how much work is involved in running a company. We've got two people and even working 16 hour days we're still swamped. Coding is like maybe 20-40% of the total work, depending on what sort of product you've got.

Start frequenting the OpenCoffee or similar in your area and start talking to people about what you're doing. There's a reasonable chance that in a couple of months you could fish up a nice co-founder.

<comment><USER10> I think you can be a one man band if you make iPhone apps (or something equivalent).

<comment><USER11> Well, I mostly meant for YC-like startups where you're shooting for large-ish returns, having to deal with fundraising and all that jazz.

</comment></comment></comment></comment><comment><USER12> get a co-founder, really.

</comment><comment><USER4> i'm curious to know too</comment>


Exercise changes the way our bodies work at a molecular level - happy-go-lucky https://theconversation.com/exercise-changes-the-way-our-bodies-work-at-a-molecular-level-73335 <comment><AUTHOR> Full text of the study referred to in the last section:

[http://www.cell.com/cell- metabolism/fulltext/S1550-4131(17)3...](http://www.cell.com/cell- metabolism/fulltext/S1550-4131\(17\)30099-2)</comment>


Orbital Sciences to Stop Using Suspect Russian Rocket Engine After Explosion - <AUTHOR> http://www.space.com/27669-orbital-sciences-rocket-explosion-soviet-engines.html

<comment><AUTHOR> The obvious question in my mind (and the reason I submitted this) is whether or not SpaceX is one of the companies Orbital is in talks with to provide launch capability in the meantime. Does anyone know if the Falcon 9 is theoretically capable of boosting a Cygnus vehicle to the ISS's orbit?

Edit: just looked at the numbers. The Cygnus is significantly smaller in volume and mass than the Dragon, so I assume now that the answer is "yes, a Falcon 9 could launch a Cygnus"</comment>


A Human-Powered Helicopter: Straight Up Difficult - <AUTHOR> http://www.npr.org/2012/10/14/160670295/flight-club-human-powered-helicopter

<comment><USER2> If a meat-powered helicopter is feasible, limited only by the endurance of the pilot, I wonder if you could power such a craft by a small, inexpensive engine, such as a lawnmower engine.</comment>


Hong Kong students take protest to virtual world - <AUTHOR> https://www.reuters.com/article/us-hongkong-protests-games/hong-kong-students-take-protest-to-virtual-world-idUSKBN1X715V <comment><USER2> Comment and Question :

1st the comment: This is absolutely frightening to me: "A role-playing mobile app game depicting the protest was suspended by Google earlier this month, while gaming giant Blizzard Entertainment suspended Hong Kong esports star Chung "blitzchung" Ng Wai after he called for the liberation of Hong Kong in a post-game interview." I am not pro or anti-China. That's not the point. What I fear is the growing reluctance of companies to do anything that may muddy their relationship with China, which amounts to effective and in-direct censorship. Did I get this right, or are there other reasons for Google's and Blizzard Entertainment's actions? Yes, yes, I know. No politics in business. But people that conduct business, in this case the video gamers, are in fact people! At one point can a celebrity, or even a regular employee, voice their political opinion without fear of reprisal. In the case of China (NBA this month), it seems it's almost never. Again, my comments are not about China. It's about how businesses stifle freedom of expression. This article could have been about Chile or Lebanon, two other countries with massive protests.

Now my question: "The students said they have submitted their game to an online video game distributor and hope to have it available to the public next month." Are there any platforms that empower these students to distribute their video game without relying on other businesses, specially big business?

<comment><USER3> With regards to your comment, you're absolutely spot on in terms of the chilling effects resulting from not wanting to anger China. That pressure will not change until the population is willing to put its money where its mouth is and stop supporting/investing in companies that cave to Chinese threats (or any other market as you mentioned).

With regards to your question...

>Are there any platforms that empower these students to distribute their video game without relying on other businesses, specially big business?

Yes and no.

Mobile devices in particular are problematic in that they're much further away from General Purpose Computers than desktops or the OS's of the days of yore.

If you wanted to get a game out, you just published the executable somewhere for downloading and marketed it well if you were out to disseminate something without profit motive.

Nowadays though, the gaming industry flourishes based on integration through various corporate entities which represent leverage points for sufficiently large and centralizable market actors.

Android and especially iOS are the most telling examples, but basically any AppStore like infrastructure that relies on market penetration to maintain it's own existence is at some point threatened.

This is where the idea of "too big to fail" comes back to bite you. When you have highly consolidated and invested in industries that are so large that they constitute large chunks of market share, yet require substantially large capital inputs necessitating the partaking of "tainted cash" you get an environment where economic hostage taking becomes a viable strategy.

Your choice suddenly becomes whether to topple/hobble those companies, potentially having disasterous effects on your labor market, or sacrificing your cultural morals to the first big market actor who takes offense.

The alternative is "rolling your own dissemination infrastructure." Which while non-trivial, has it's costs associated with it primarily in the sense of "what sets of tools are at my disposal to get something out there?"

When you have an actor such as China clamping down on for-profits in order to prevent them from providing aid, that essentially means starting at the ground floor. All you can really assume is there for you without the threat of a lawyer knocking on your door with bad news from a vendor is Free Software, and whatever smaller actors you can gather who aren't yet big enough to be adversely effected by your Adversary.

Not an enviable position to be in to be quite honest.</comment></comment>


Show HN: Particubes - <AUTHOR> http://appstore.com/particubes <comment><AUTHOR> Not mine, but just found out this cool easy voxel art iOS app.</comment>


Political Theory and Real Politics in the Age of the Internet - <AUTHOR> https://www.sms.cam.ac.uk/media/1918927 <comment><AUTHOR> This is extremely dense and extremely fascinating. I also recommend his podcast, Talking Politics (which has a substantial UK focus, but deals with America and world politics too).

[https://www.acast.com/talkingpolitics/](https://www.acast.com/talkingpolitics/)</comment>


Ask HN: What are the performance gains using native C++ instead of C#? - <AUTHOR>

Especially in video games <comment><USER2> Depends on the programmer.

Both C# and C++ can be used to make a video game, they can be used to make a 3D video game. Heck you could almost certainly write a well selling commercial game in C#. I mean Minecraft was written in Java and look how well that has done...

The reason people like C++ is because you can optimise it down to near its optimal speed at key choke points in the code. It is kind of like the 95% rule, 95% of the time C++ and C# will run at similar speeds, but it is that 5% where C++'s versatility really matters.

<comment><AUTHOR> But minecraft dosen't look photorealastic; I mean, the engines like luminous, unreal engine 4, id tech 5.

Can C# or Java be optimised to run these kinds of codebases with near similar efficiency?

<comment><USER2> The thing about these large industrial game engines is that they're build from very efficient libraries already written in C/C++. So the optimisation has already happened.

You could write a "game" in C# that uses the native broker to call into the C/C++ libraries and generate something remotely similar to what a modern game engine does, but there will always be overhead in doing so.

Efficiency isn't really an interesting discussion topic in this area to be honest. What is interesting is the fact that almost all of the code already written for games is in C/C++.

You want to use a library? It will be C/C++. You want full access to DirectX? C/C++. You want access to all of the OS's underlying APIs/data-structures? C/C++.

Even if C# was entirely as efficient as C/C++ (and as I said 95% of the time it really is) - it doesn't actually matter, because for the last twenty years programmers have been pumping out game libraries for C/C++, not for C#.

C# and many other languages have a chicken/egg problem: Nobody wants to write game code for the language because nobody is writing game code for the language.

<comment><AUTHOR> Is there any possibility of the video game industry market shifting toward C# in forseeable future?

I emailed Aras, lead graphics programmer at Unity technology. He replied that C++ in video game industry, esp in game engines is indispensible, and it will remain the standard in long forseeable future.

I also sent an email to Mike Abrash at valve, and he replied that there is no movement away from C++.

<comment><USER2> > Is there any possibility of the video game industry market shifting toward > C# in forseeable future?

No chance at all.

It offers nothing that C/C++ doesn't already. It has no game libraries written for it at all. It is largely unsupported on non-Windows platforms (e.g. Android, Linux, Mac, etc).

Best case scenario: The XBox 720 allows "apps" to be written in C# and sold to end-users. So C# becomes the de-facto indie game language of the XBox 720 (as in mini-games, not full AAA titles).

</comment></comment></comment></comment></comment><comment><USER3> The problem with C# and Java, among a number of things, is that you can't implement some LOD algorithms without adding considerable GC overhead. Example: <https://graphics.llnl.gov/ROAM/roam.pdf>

Modern video games will use all extensions available, this can't be done directly in C# nor Java, you need some native code to do that. Example: <http://glew.sourceforge.net/>

Also, you want to make you game portable, if you write a physics engine in C++ you could run it the it is on iOS, Android, Windows, Mac and most consoles, try that with C# or Java.

Th use of state-of-art algorithms and extensions provided by gfx card manufacturers add zillion-folds of performance gains. The difference is not on frame rates but complete impossibility of doing some more advanced effects without those extensions. If you are building something like minecraft, you should be just fine on C#, Java and maybe JavaScript. The trend now seems to be less focused on graphics and more on the fun itself.

People also use C/C++ because everybody does and you probably learned game development using GLUT or something like that.

:D

(edited last paragraph for clarity)

</comment><comment><USER4> Assuming that you're fluent in both languages to the same level And that you're talking about raw performance (and therefore ignoring anything to do with development time, not having to track down dangling pointer bugs, having multi platform codebases, individual platform support etc).

\- There'll be little or no perf difference for GPU between the 2 as the underlying drivers will be the same. \- C# can call c++ which can in turn call assembly so if you really need bare to the bones performance you could probably manage it....

So really...for me it really comes down to garbage collection, currently you have no real control over when it happpens and how long it takes; and that really is a killer in games. If you have 16ms to render a frame and then suddenly GC kicks in for 10ms then you might as well pack up your bag and go home.

If I could specify when and how long the GC goes for with some sense of control over it then I'd love to use C#.

But without that control it's a pain in the bottom.

</comment><comment><USER5> Unreal Engine uses UnrealScript. No idea about id tech. Unity is c#.

There are speed differences, but this is greatly dependent on what you are doing and unless you are Epic or iD, I doubt it will matter. And if you are asking this question, I really doubt you will be able to use C++ to get the optimisations those engines have.

You can take a look at <http://benchmarksgame.alioth.debian.org> to get an idea, c# in the mono runtime is about 3-4 times slower than C (from other benchmarks, the mono runtime is slower than the microsoft one)

<comment><AUTHOR> Unreal Engine is writern in C++. So is Unity3d. They just provide an abstraction layer to US and C# respectively....

<comment><USER5> Yes, and as I mentioned, I really doubt you are at the level where those performance gains matter. If you use c# you will be working with DirectX, which most means much of the heavy lifting is done there. Unreal and Unity have teams of programmers working trying to squeeze the last bits of performance (Unreal winning there for sure) of every platform. Which leads to my other point. I think the main reason for anyone (unless we are talking 50+ people team on the engine) to use c++ is portability NOT speed.

</comment></comment></comment><comment><USER6> Other people have discussed the speed, but one aspect of performance that people often ignore is memory usage. In C# (or at least the most common implementation), there is a per object overhead of at least 8 bytes on 32-bit platforms and 16 bytes on 64-bit platforms. C++ implementations typically have no overhead for objects without virtual functions and 4 or 8 bytes of overhead for objects with virtual functions. This means that a program written in C++ could potentially use a much lower amount of memory than one written in C#.

</comment><comment><USER7> The advantage of C++ is that you can completely control memory layout. Because of the way modern CPU's work, making sure that data is aligned to minimize cache misses is super important when writing high performance code. But it is only important for maybe 5% of the code (the low level vertex and texture management code). The rest you can write in _almost_ any language.</comment>