TeXmacs 1.99.1 - <AUTHOR> https://github.com/texmacs/texmacs/releases/tag/v1.99.10 <comment><USER2> Example document written with TeXmacs: [https://edoc.ub.uni- muenchen.de/15166/](https://edoc.ub.uni-muenchen.de/15166/) (neuroscience PhD thesis)

<comment><USER3> Nice!

</comment></comment><comment><AUTHOR> We have videos:

[http://texmacs.org/tmweb/home/videos.en.html](http://texmacs.org/tmweb/home/videos.en.html)

It is worth your time.

<comment><USER4> I used texmacs not so long ago and as a guile user I was a bit thrown off by the ancient guile version. Are there any plans to migrate to the newer and faster 2.2 or later 3.0?

I am not in any position to request things, but I got bitten a bit by the fact that macro expansion happens at every startup, and due to me loading a lot of extra modules for my own convenience texmacs took a long time to start.

<comment><USER5> In the meanwhile you can just load those modules lazily, TeXamcs does the same for some of its functionality, so you can look at the code to see how to do it. Compilation will help this but for the moment the port to Guile 2 in on an early stage and I cannot tell you exactly how long will take. Hopefully less than one year from now. We are all quite busy otherwise. Help is appreciated, especially from scheme programmers.

<comment><USER4> I am not a programmer, but I do write some scheme from time to time. If you end up in #guile on freenode I can try to help you the best I can.

<comment><USER5> Thanks. I will show up when I'm going back to the port. Right now real life is calling :) but I plan to resume working as soon as possible, in few weeks.

</comment></comment></comment></comment></comment><comment><USER6> For interested people: there is at the moment a proposal for a StackExchange website [https://area51.stackexchange.com/proposals/121978/texmacs](https://area51.stackexchange.com/proposals/121978/texmacs)

Especially upvotes to questions are important, as enough people have joined to pass the first stage, but not enough questions have been upvoted.

</comment><comment><USER5> Please remark the main web page at: www.texmacs.org

</comment><comment><USER7> Do many people use TeXmacs? Now that you can export markdown and org-mode to LaTeX, TeXmacs doesn't seem to provide much value. I use org-mode with embedded LaTeX for many documents and it works really well for standard stuff. You still are probably going to want to use LaTeX (or even just TeX) for complex documents like resumes and the like though.

I've tried using TeXmacs once and it never really worked well. Maybe because I already can write LaTeX reasonably quickly, but using it always felt slow and clunky.

<comment><USER8> I tried exporting org-mode to latex in the past and found it extremely buggy. I don't really see a reason not to just directly use LaTeX instead.

<comment><USER9> I have co-authored several academic articles in Org mode with exporting to LaTeX. The main reasons for choosing Org were that (i) we wanted to include the analysis code, table generation and figures in the same file as the manuscript, and (ii) the workflow was Emacs-centric. The Org markup was simple, readily allowed for inserting most LaTeX, had nice tables, but was fiddly for some more complex tasks. I recently needed to convert one of the manuscripts from Org to Word - pandoc was very helpful.

We could possibly have used TeXmacs, R markdown (especially with RStudio), some variant of noweb, or even Jupyter with conversion to LaTeX using pandoc.

</comment></comment></comment><comment><USER10> One of the finest pieces of software out there. (It is thought to be inspired by emacs and TeX, but the workflow bears little resemblance to using either of them.)

</comment><comment><USER11> TeXmacs is a really interesting and fun product. Tried several times. Love its idea. There are lots of bugs last time I tried. Glad to see it is moving forward.

<comment><AUTHOR> I have to admit that GNU TeXmacs is buggy.

And I've fixed some of them by myself or with the help of other contributors.

Please report bugs to [https://savannah.gnu.org/bugs/?group=texmacs](https://savannah.gnu.org/bugs/?group=texmacs)

And we will fix it as soon as possible.

For the next release: TeXmacs 2.1, we mainly focus on bug fixes.

</comment></comment><comment><USER12> TeXmacs is a beautiful piece of software, much better (in my opinion) than TeX. I wrote both my bachelor's and master's thesis in TeXmacs, and used it for years for keeping a research notebook, for homework assignments during undergrad, and for general note taking. I am delighted to hear that it is still being developed. Thank you to the developers for their hard work!

<comment><USER13> How does TeXmacs compare to LaTeX?

<comment><USER5> LaTeX is a typesetting system, so usually you use it together with an editor and a viewer. TeXmacs is _one_ program which does it all in such a way that you can edit the typeset document directly and that what you see on the screen is _exactly_ what will be on the final PDF. These features make it unique. Typesetting quality is comparable to TeX in the sense that you can well mistake a TeXmacs document with a LaTeX one. TeXmacs has an integrated picture editor and a presentation mode, and you can have interactive sessions in your document e.g. with software like R, Python, Scheme, Axiom, Reduce, etc... Actually you can even run code inside TeXmacs to produce images with libraries like TikZ/FeynMF/DraTeX/Asymptote/Graphviz whose functionalities have no equivalent (yet) in TeXmacs.

<comment><USER13> Then why is the grandparent poster comparing TeXmacs to TeX?

<comment><USER5> Since he seems to have used it already I also assumed (s)he knew the difference and that with TeX it was meaning a TeX-based workflow. But I wanted to make clear the difference for newcomers since TeXmacs is often misinterpreted as just a TeX frontend, which is not the case.

</comment></comment></comment></comment></comment><comment><USER14> What are the main differences between TeXmacs and LyX?

<comment><USER5> Many. Among which TeXmacs is a _structured_ editor. You manipulate the structure of your document, from chapter, to sections, to environment, down to fractions and brackets all is an environment, which means that you cannot have, for example, (unwillingly) unmacthed brackets. Then TeXmacs contains the UI, the typesetter, an interface to other programs like computer algebra systems, an _internal_ image editor, an _internal_ versioning system, all in \sim 170MB (without other external dependencies). An is extensible via Scheme (much like Emacs is extensible).

<comment><USER15> LyX is also structured in the sense you describe.

But somehow I never realised that TeXmacs is actually not TeX at all, I assumed it was another front-end. How on earth did it get such a confusing name?

<comment><USER5> Historical reasons. Initially was inspired by TeX and Emacs. The project started in 1998 so now is hard to change the name. For the moment there is no consensus on a better name, so we stick with it and just try to make it clear from the outset.

</comment></comment></comment></comment><comment><USER16> Well done guys; your efforts are really appreciated. I use LaTeX for everything usually through Rnw files so that I can embed R all over the place. For me, I think some amount of literate programming functionality would be essential before I could do much with this.

Does it support different types of layouts, and the sort of type-setting flexibility one might be used to with LaTeX?

<comment><AUTHOR> It is a pleasure to write math in GNU TeXmacs.

One of my friends write the draft of a paper in GNU TeXmacs, and then export it to LaTeX, finnaly, he polish the paper in LaTeX.

Other guys use TeXmacs for Maxima/FriCAS/Octave.

</comment></comment><comment><USER17> I used this about 10 years ago to write maths-heavy papers. It was really fun. My favourite feature was you could tab through alternative mathematical symbols: type an equals sign, then <tab> until it became \equiv or approximately equal or whatever.

The project seemed to stall and I moved on to LyX. Nice to see it is back in business.

<comment><AUTHOR> The project needs more developer.

We are trying to make it back to Ubuntu/Debian.

> The project seemed to stall and I moved on to LyX. Nice to see it is back in > business.

Actually, Joris writes code for GNU TeXmacs on a regular basis. But he did not update the main site before. The old-styled main site (before) delivered the wrong message.

</comment></comment><comment><USER18> Good to know that this exists. I love LaTeX so much, and I've been using Overleaf for my projects.

As a student in Brazil, all my assignments have to be formatted in the ABNT norms. Doing it on Word is so annoying, so I used a LaTeX framework (abntex2) so I can only focus on the content and not on the formatting.

<comment><USER5> TeXmacs has also a macro system, in principle you could write a package style for this format.

</comment></comment><comment><USER19> A small thread from 2016: [https://news.ycombinator.com/item?id=11103012](https://news.ycombinator.com/item?id=11103012)

</comment><comment><USER20> "TeXmacs-1.99.10" can't be opened because it is from an unidentified developer." Why not become an identified developer?

<comment><AUTHOR> You are talking about installing and opening GNU TeXmacs on macOS?

We may improve it later. For now, just trust it and be a happy TeXmacs user.

</comment></comment><comment><USER21> Typo alert: Title here says 1.99.1 but it's actually 1.99.10.

<comment><AUTHOR> Sorry, this is my title:

GNU TeXmacs v1.99.10 is released!

But for some reason, the title was cut.

<comment><USER22> In few hours after you submit any news on HN, you could edit titles (that automatically modified by HN)

</comment></comment></comment><comment><USER23> The keyboard latency makes this program completely unusable! I've been following the project for the last two decades hoping someday it would be useful to touch typists who type around 120 wpm. Until then ... I just keep using emacs + auctex. Sure looks nice though.

<comment><USER23> NB: I tried both slowphil's custom build on github and the native windows binary downloadable from the main web site. Typing lag (estimated at 500-800ms) is totally unacceptable in both instances.

<comment><USER5> I can understand, but please appreciate that the task the program has to perform to process typing is quite complex, especially because some delay is needed to allow keycombinations. Anyway we plan to improve responsiveness in the future. Thanks for the feedback.</comment></comment></comment>


ES6 generators in depth - <AUTHOR> http://www.2ality.com/2015/03/es6-generators.html

<comment><USER2> A very in-depth post!

As Python has had generators for years already there is plenty of resources out there for the curious that translate well to ES6[0].

[0] [http://www.dabeaz.com/coroutines/](http://www.dabeaz.com/coroutines/)

<comment><USER3> His work is great! Linked to from the blog post...

</comment></comment><comment><USER4> Here is another deep explanation of generators, by Kyle Simpson:

[https://github.com/getify/You-Dont-Know- JS/blob/master/async...](https://github.com/getify/You-Dont-Know- JS/blob/master/async%20&%20performance/ch4.md)

Both of them are really good, and come at it from slightly different angles.

<comment><AUTHOR> And if you follow them on Twitter there are some good conversations back and forth. ;)

</comment></comment><comment><USER5> There is a GitHub repo too --> [https://github.com/rauschma/generator- examples](https://github.com/rauschma/generator-examples)

</comment><comment><USER6> I want to like generators for asynchronous code, but the one thing that disappoints me about generators is how awkward it is to write code that does the same as an async.auto

I use async.auto because of the ease of adding new asynchronous functions that depend on one or more other asynchronous functions having been executed beforehand.

Simply put you can break up a task into steps that depend on each other, and the steps run whenever all their dependencies have been executed. You don't have to worry about whether the functions run in series, or parallel, you just define that one function depends on another and it just works.

I have yet to see a promise example, or a generator example that had the combination of maintainability and legibility that an async.auto has. What I really want to see is a reasonably complicated task that has say 8 or more asynchronous steps mocked out in generators. Say something like:

Post a social item

1) Write social item to database

2) Write reference to social item in feed of person who posted item

3) Fetch friends of user who posted item

4) Fetch notification settings of each friend

5) Add a reference to the social item to each friend's personal activity stream

6) Based on notification settings of each user send an email alerting each friend of the event

7) Invalidate activity stream cache key for poster, and friends

In this flow:

1 & 3 can run in parallel at the start as they don't depend on anything

2 & 5 can run as soon as 1 is finished as they depend on the ID of the object in #1

4 & 5 can run as soon as 3 is finished as they depend on the friend list

6 can run as soon as 4 is done as it needs to know who to email and who not to email

7 can run as soon as 2 & 5 are done as the streams must be updated before the cache can be cleared

With async.auto it is easy to create 7 async functions and define the dependencies, then sit back and watch it run efficiently and magically. Later if you need to add more actions it is easy to insert another function, define what it depends on and see it working.

I still don't understand how to do something like this with generators, and none of the examples I've seen so far is complex enough, nor do they seem to solve this problem in a format as clean as async.auto

</comment><comment><USER7> I didn't realize they doubled as both iterables and observables. Being able to pass a value in is a useful trick, although it looks a bit annoying to forward passed in values *due to error/return requiring try-catch shenanigans).

</comment><comment><USER8> Amazingly in depth post!

I'm sad when programming languages have to reach for new keywords and syntax to add features.

`function*` is difficult to google for, and doesn't have any human readable hints for somebody to understand.

<comment><USER3> JavaScript is constrained by backwards compatibility. That's why the ideal solution, the keyword `generator`, wasn't possible.

<comment><USER9> which is strange, considering the `async` keyword in upcoming versions.

<comment><USER3> AFAICT, `async function` is OK, because you combine a keyword with something that is contextually turned into a keyword. With a single non-keyword that isn't possble. For example, the following code (a hypothetical ES6 anonymous generator expression) could be an ES5 function call followed by a code block.

        generator (a, b, c) {
            ···
        }</comment></comment></comment></comment>


Getting controversial iPhone app rejected an easy way to promote your accepted apps? - <AUTHOR>

Controversial app rejections tend to make mainstream news.  So why not submit a controversial app on purpose to direct traffic to your accepted apps? <comment><USER2> Or you can make a worthwhile app?

Seriously, does that traffic translate into purchases? The majority of people who follow these releases are developers themselves. That seems like a lot of work to raise a little publicity. That effort would be put to better use relying on polishing your main app and spreading the word the old-fashioned way.

<comment><AUTHOR> Have you had much success recently with the "old-fashioned way"? If so, could you be specific about which method(s) in particular were effective?</comment></comment>


Compiler Bugs Found When Porting Chromium to VC++ 2015 - <AUTHOR> https://randomascii.wordpress.com/2016/03/24/compiler-bugs-found-when-porting-chromium-to-vc-2015/ <comment><USER2> _And I have to say that the Microsoft team was amazing. They were very supportive, and helpful, and it was clear that they really wanted VC++ 2015 to be as good as possible_

Rather anecdotical of course, but I had the same impression. File a _proper_ bug report and you get a reply in the next couple of days, typically starting with _thanks for the excellent bug report ... will try to get the fix in the next update_. Found a couple of bugs in VS2012 and VS2013 and they were resolved in either the next update or in the next version. Practically speaking it usually took somwhere between 1 - 3 months, which isn't too bad all things considered. Like, I've also had much worse experiences with some competitors where I'd file a report including a possible fix (something like 'change <= to <' in soure x line y), and 6 months later there even hasn't ben a reply let alone a fix.. Anyway with VS2015 it even seems like they cranked it up a notch beacuse the bugs I encountered already had a fix in the update I didn't manage to install yet :]

<comment><USER3> > _Like, I 've also had much worse experiences with some competitors where I'd > file a report including a possible fix (something like 'change <= to <' in > soure x line y), and 6 months later there even hasn't ben a reply let alone > a fix._

 _cough_ Apple _cough_ ...

A few years ago I reported an issue for iOS that _never_ got a response. When I made it to WWDC 6 months after reporting it, I went the support session and ended up running into the guy that wrote the bug. He was aware of the issue, it had been reported a few times, and he gave me a workaround until they released the fix. It was nice to finally get a resolution, but the whole process was seriously fucked up. After the fix was released in the next version of iOS they closed my bug report without a single comment.

<comment><USER4> Sounds like they're using an internal tracker not publically accessible... making the public one more of a pony show than anything.

Sadly, this is common for a lot of places (they're afraid internal discussions would get negative attention if made public in some tracker).

<comment><USER5> > (they're afraid internal discussions would get negative attention if made > public in some tracker)

Mostly, they're afraid internal discussion would leak super-seekrit info about their product pipeline. You know how anal they can be about their product announcements.

I mean, they get worked up about leaks like this: [http://arstechnica.com/apple/2016/03/unreleased-12-inch- macb...](http://arstechnica.com/apple/2016/03/unreleased-12-inch-macbook- early-2016-shows-up-in-os-x-server-update/)

</comment></comment></comment></comment><comment><USER6> At the bottom of the article Bruce mentions the slashdot comments went downhill. This is the first time I've visited slashdot in years and I'm sad to see an old haunt turned so bad. The equivalent reddit thread is significantly more interesting, positive, and hacker-ish: [https://tech.slashdot.org/story/16/03/25/141256/chromium- bei...](https://tech.slashdot.org/story/16/03/25/141256/chromium-being-ported- to-vc-scrubbed-of-compiler-bugs#comments) [https://www.reddit.com/r/programming/comments/4bvswj/compile...](https://www.reddit.com/r/programming/comments/4bvswj/compiler_bugs_found_when_porting_chromium_to_vc)

<comment><USER7> I overstayed at Slashdot. I kept visiting it probably due to a long habit of it being the first news site I'd check while code was compiling. Just ingrained in muscle memory.

Then, one day, I realized most discussions had turned vile. Cynical rants that would find fault with absolutely anything, and I would feel dirtier reading it.

It's really a shame, a bit like seeing an old friend that ended up with wrong choices in life.

<comment><USER8> > Cynical rants that would find fault with absolutely anything.

I think that is just comments on the internet in general, everywhere has them, I had to work hard at stopping it bothering me, you can't bath in negativity and not get wet.

</comment></comment></comment><comment><USER9> "I generally don't consider my own software adequately enough tested until its tests have turned up a bug in a compiler/toolchain."

[https://www.reddit.com/r/Bitcoin/comments/2rrxq7/on_why_010s...](https://www.reddit.com/r/Bitcoin/comments/2rrxq7/on_why_010s_release_notes_say_we_have_reason_to)

<comment> Ono-Sendai You're not going to discover compiler bugs if you stick to language features and compilers that have been thoroughly trodden, and don't push the limits in any special way (massive functions etc..).

<comment><USER10> For an example of what Greg Maxwell might consider "adequate testing", see [https://www.ietf.org/proceedings/82/slides/codec-4.pdf](https://www.ietf.org/proceedings/82/slides/codec-4.pdf) (he is the one responsible for all of the testing described in that presentation).

Slide 20 lists no less than 8 toolchain bugs found with a codebase of less than 50k lines of C code (no C++), using only thoroughly trodden language features, without "massive functions" or anything else. Admittedly 4 of those bugs were in pre-release versions of gcc, but the rest weren't.

<comment> Ono-Sendai I've heard ICC is pretty buggy. Tinycc seems a bit obscure. pre-release versions of GCC as you mentioned are not well-trodden.

<comment><USER11> _I 've heard ICC is pretty buggy._

Perhaps historically ICC was buggy, but I don't think it's true anymore. I often test highly optimized C code with GCC, Clang, and ICC, and anecdotally I'd say that the likelihood of hitting a compiler bug when compiling for a current Intel processor is about the same for each.

For me, crashing bugs and true miscompilation are rare with all three, but come up occasionally. Performance differences are usually within +/\- 20% on microbenchmarks, with each having about equal chances of being the fastest or slowest.

</comment></comment></comment></comment></comment><comment><USER12> > _" I worked on one project where in addition to finding several compiler > bugs I also found a flaw in the design of the processor, but that's an NDA > tale for another day…"_

Yeah I really want to hear this story; whens the NDA expire or does it never? :)

<comment><USER13> I wonder if he worked with early UltraSPARC chips. There were tales of an interesting bug report from the NSA. It's been 20 years, so I probably have every detail wrong, but it was something like a missed implementation of a version of ROTL that the CPU was basically emulating at several orders of magnitude performance penalty. No other customer found it.

</comment></comment><comment><USER14> It would be nice if MSVC will create something like clang-cl [1] to handle --options like others do, for easier compiling of pure Makefile opensource projects.

[1] [http://clang.llvm.org/docs/MSVCCompatibility.html](http://clang.llvm.org/docs/MSVCCompatibility.html)

<comment><USER15> Why use MSVC at all when Clang has a perfectly good gcc-style driver? There's also this [http://git.savannah.gnu.org/cgit/automake.git/tree/lib/compi...](http://git.savannah.gnu.org/cgit/automake.git/tree/lib/compile) wrapper script which is useful once in a while for invoking MSVC from inside Cygwin or MSYS (though MSYS2 broke this, last I checked).

<comment><USER14> We (radare2 project) already being built by gcc, clang and tcc. Every time new compiler support added it reveal some new errors. This is why we have interest in trying MSVC too. Thank you for the script!

</comment></comment></comment><comment><USER16> For over a decade I've felt that Visual Studio is consistently the best product that Microsoft has released. Which makes sense since many if not most of their developers rely on it to make other product.

<comment><USER17> Yup, MSVC is the bar that I hold every other IDE to. IntelliJ comes close but all others (Eclipse I'm looking at you) fall far short.

</comment></comment><comment><USER18> You haven't programmed in C if you haven't found a compiler bug.

</comment><comment><USER19> I'm currently working to create a project demonstrating a Visual Studio 2015 C++ compiler crash that exhibits only when using the Inline Function Expansion optimization. The Visual Studio team has proven very responsive.

</comment><comment><USER20>
        char key[5] = { 0 };

        Simple enough – this is supposed to zero the entire array, but instead it only zeroed the first four bytes.

In C++ and on the stack "{0}" is not required as memory is zeroized. On heap (in a new) "{0,0,0,0,0}" should be used (or memset()). Not sure about that in C.

My understanding is that it's not a bug.

<comment><USER21>
      #include <iostream>

      int main() {
          int nope_not_initialized;
          std::cout << nope_not_initialized << "\n";
      }

      cc1plus: warnings being treated as errors
      In function 'int main()':
      Line 5: warning: 'nope_not_initialized' is used uninitialized in this function

( [http://codepad.org/6xrFQZuL](http://codepad.org/6xrFQZuL) )

Edit: And a C version. Amusingly, it will print "0" if I only use one printf statement:

      #include <stdio.h>

      int main() {
          int nope_not_initialized;
          printf("%d", nope_not_initialized);
          printf("%d", nope_not_initialized);
      }

      -142791388-142791388
      Exited: ExitFailure 10

( [http://codepad.org/TgGfL7QW](http://codepad.org/TgGfL7QW) )

Edit x2: Apparently "implicit return 0 from main" is only a thing in C++? And here I'd assumed it was some silly carryover from C...

( [http://codepad.org/JIH9ZteL](http://codepad.org/JIH9ZteL) )</comment></comment>


Sysadmins see evidence that they have been hacked by GCHQ [video] - <AUTHOR> http://www.spiegel.de/video/nsa-und-gchq-hoeren-telekommunikationsunternehmen-ab-video-1521330.html <comment><USER2> This is the right way to do it. Great job Spiegel (or whoever worked on this piece). Putting real people on screen, real faces. Showing emotion, showing them swallowing knots when they see their names on the screen of "tasked" engineers.

I think here on HN and other tech and privacy forums we understand what is happening. Unless there is a reporting like this, it will be a bit harder to engage a wide audience. Telling the proverbial grandma about "PRISM" or "they are listening to everyone" is not going to quite work. What works is to do this -- showing one particular grandma with a name, address, life story and showing how maybe her recipe for baking cookies is now logged in Utah's NSA's headquarters in room 5B, on storage node 18Z and so on.

<comment><USER3> Produced by Laura Poitras. (just watch it until the end and you'll get the other names)

<comment><USER4> Ten bucks she is "tasked".

<comment><USER5> Poitras is actually the shadow of this whole story. She was the journalist Snowden reached out to when Greenwald was unresponsive, was there and filmed the first Snowden interviews, was responsible for the Bill Binney story in the NYT a few years back[1] and has been on the US Government shit list for quite a few years[2].

[1] [http://www.nytimes.com/2012/08/23/opinion/the-national- secur...](http://www.nytimes.com/2012/08/23/opinion/the-national-security- agencys-domestic-spying-program.html?_r=0)

[2] [https://en.wikipedia.org/wiki/Laura_Poitras#Government_surve...](https://en.wikipedia.org/wiki/Laura_Poitras#Government_surveillance)

<comment><USER2> I think you might have misread what the gp meant by "tasked". It was taken directly from the video. In the video "tasking" meaning being selected, followed (presumably virtually only), monitored and spied on.

</comment></comment></comment></comment></comment><comment><USER6> Satellite communications providers, especially those offering L-band (mobile) services, are really the low hanging fruit of the SIGINT world. They're pretty much only used by "interesting" people due to cost, in areas which are inaccessible otherwise (non-permissive to HUMINT, etc.)

That they tend to be run by technically incompetent people, using expensive black box hardware they don't understand, and with multiple levels of indirection between end user and the Internet (transponders, ground stations, facilities, virtual network operators, ...) makes it all much more vulnerable.

Combine that with price sensitivity (so subsidized government stuff can be cheaper, and legal expenses unacceptable), and a highly regulated environment (ITAR + various spectrum licensing and launch regimes), and it's a perfect storm.

The only more interesting target would be "satellite comms network dedicated to high value international payments".

(Disclaimer: I started/ran a satellite communications and wireless provider, and worked for or with a bunch of others.)

<comment><USER3> > (Disclaimer: I started/ran a satellite communications and wireless provider, > and worked for or with a bunch of others.)

Which means you are under active and targeted surveillance. Great to see you put encryption contact info in your profile.

<comment><USER7> rdl has a few other fun reasons to have attracted the Eye of Sauron. Like that one time when he was the only inhabitant of an entire country.

<comment><USER3> You can't just stop after telling that… Only inhabitant of an entire country? How can it be?

<comment><USER7> One account is in [http://works.bepress.com/cgi/viewcontent.cgi?article=1035&co...](http://works.bepress.com/cgi/viewcontent.cgi?article=1035&context=james_grimmelmann)

You might notice some discussion there of what counts as a country.

<comment><USER3> Thank you both!

</comment></comment></comment></comment></comment></comment><comment><AUTHOR> Stellar PCS is a german ISP company that is specialized in bringing internet access to inaccessible regions via satellite. Clients include research stations or oil rigs. In this video you can see a SPIEGEL journalist showing them evidence that the NSA has hacked their network for the first time.

<comment><USER8> Clearly it's for counter terrorism, and not corporate espionage /s

<comment><USER3> Yes, clearly, yes. Because USA do not have ANY economic interest in Africa and Middle East, and whatever these countries have as a government it doesn't change a thing for them. I mean, it's not like they have oil to sell and can alter the world's economy with a simple statement.

<comment><USER9> This, as much as the domestic spying stuff bothers me, this uber-left uber- pacifist view of things is asinine. China, Russia, Iran, etc aren't stopping their SIGINT or cyberwarfare programs and neither should we.

I also don't care if Germans are our "allies." 9 months ago Russia and Ukraine were best friends, now Russian soldiers are blowing away Ukranian civilians with total impunity. Shit changes. Shit gets real quick. Being at a information disadvantage can lead to serious consquences.

I think we live in a time too used to peace and as we can see from recent events, that time is now over. The far left's obsession with pacifism and the far right's obsession with isolationism are just impractical. This just causes conflicts that need to happen to be put away and ignored which leads to larger conflicts later. For example, the US and Iraq should have worked together when ISIS took Fallujah MONTHS AGO. The US should not have caved into pressure from the EU and Russia to not take out Assad. Instead, we chose the path of politics and sticking our head in the sand and an entire region just became destabilized. On top of it, petty dictators like Putin see our weakness and use it against us by invading his neighbors, knowing he'll only receive a slap on the wrist.

Downvote away, but we need SIGINT, now more than ever.

</comment></comment></comment></comment><comment><USER10> I'm shocked that they are securing such important routers with a username/password combination like horizon/h0riz0n. Such a system should be protected by public and private keys.

<comment><USER11> I assumed that this 'customer' was a trojan horse, put there by the NSA.

Well, that's what I'd do anyway.

It goes like this - Stellar gets offered a big hosting contract that they aren't going to turn away. They don't do any due diligence on this new customer, and if they did, they probably hit a couple of USG Cayman Islands dead-ends anyway. They install this company's servers inside their own network.

Boom... headshot.

<comment><USER12> You are doing something very important here. Many people forget that the guys running the show at the NSA are, in all probability, at least as savvy as any one of us. For this reason, it is always a good starting point to ask oneself "What would I do if I was in their shoes?".

A prime example of how this thinking can be applied is the TrueCrypt fiasco. Ask yourself: If you were the group leader at the NSA tasked with TrueCrypt, would you have your undelings doxx the authors? Would you then try to lean on the authors?

If I was a group leader at the NSA, I certainly would.

<comment><USER11> > Many people forget that the guys running the show at the NSA are, in all > probability, at least as savvy as any one of us.

Indeed, but I would add that these people are more than just savvy. Many of these people have been picked out because they are smarter than the average bear. They've also be brain-washed into the mold to believe that if you aren't inside, then you are the enemy, or the friend of my enemy, or a potential "task".

Many of these guys (and girls) are converts - young hackers who have been caught hacking and have been given the option to serve jail time or join the cause. Its an easy sell to young impressionable minds who want to be a hacking James Bond.

More importantly, these guys are hacking targets across the world with a remit; a licence to hack if you will. If you or I go out hacking random companies for fun and profit, we'll get a 5am dawnraid knock-knock visit and spend a couple of years 'rooming with Bubba'.

These guys can do what they want without the fear or stress of that 'Sword of Damocles' hanging over their heads. They have a free reign, and they are smart. They also have the feeling that what they are doing is right. That makes them way more dangerous than you or me.

Thanks the hat tip nevertheless. I grew up in a government security type environment. These things rub off on the kids. Somehow you learn to evaluate risk, locations and people very quickly in this kind of environment. I guess it is useful in some ways. It also makes you a constant analyst, which tires the brain somewhat, but you see things that others don't.

</comment></comment></comment></comment><comment><USER13> The reporter said "Develop and task key engineers" means surveillance but I don't think that's right. Task can be used multiple ways but I think that line is talking about recruiting key engineers as agents, probably using bribery.

I could be wrong but I thought Ali had an extremely guilty reaction. As if he was waiting for the reporter to accuse him of being an NSA asset. Which he very well may be.

<comment><USER14> So aside from this video, what concrete proof do you have to make your wildly biased claim that he may be an NSA asset? I love your dripping tone of "I'm not _saying_ he's an asset, but seriously, look at that guilt on his face. Again, not directly saying, but look at him." Maybe you're an NSA asset. You may very well be.

Let's keep personal attacks down and talk about the issue at hand rather than go for character assassination.

<comment><USER15> There wasn't any personal attacks or character assassination, he or she was just stating what his or her interpretation of their reactions was.

I actually thought the same. I kept thinking the whole point of the clip would be that they would see their own names and how they had been recruited. I, too, thought 'tasked' meant bribed/coerced.

... that's not character assassination, that's just like my opinion, man.

</comment></comment></comment><comment><USER3> Impressive. However I regret that we don't see when they commented with "Fuck". Not for the word, clearly, but because the face that went with it should have been really powerful.

I hope that now sysadmins from all over the world know that they are subject to NSA surveillance. If you are a sysadmin, please read:

[https://firstlook.org/theintercept/2014/03/20/inside-nsa- sec...](https://firstlook.org/theintercept/2014/03/20/inside-nsa-secret- efforts-hunt-hack-system-administrators/)

You could easily be a target for TAO:

[http://www.spiegel.de/international/world/catalog-reveals- ns...](http://www.spiegel.de/international/world/catalog-reveals-nsa-has-back- doors-for-numerous-devices-a-940994.html) [https://news.ycombinator.com/item?id=6979457](https://news.ycombinator.com/item?id=6979457)

If you are a sysadmin, they are after YOU.

<comment><USER16> I think we can see that "Fuck" on their face, just look at the big swallows of the guy and he even says "Oh my god" in the video.

</comment></comment><comment><USER17> Is it really possible to protect yourself and your network from these types of attacks? Any company with sysadmins or internal security teams is extremely out gunned against someone like the NSA (it is almost comical) [1]. From the perspective of a sysadmin, who has worked in startups, small companies, a university, and several government departments, I can firmly tell you that, we are not in the same league! Sure we take the yearly security courses, use best practices, harden machines and infrastructure, but after reading these articles.. we are sitting ducks. If the NSA is in bed with US based network gear providers, they can simply own the network and telecom infrastructure (via build in backdoors), and you do not even know they are there, because they side step the normal exploit channels [2].

Probably the best way to describe this, is to compare security and pro sports teams. From what I have read, the NSA is a top tear team winning championships across the globe, with billions in research and development, and thousands of highly trained athletes, living and breathing this day in and day out. Yet, they are matched up against a local beer league who likes to play casually Thursday nights. Who do you think is going to win?

Go read the "A Look at Targeted Attacks Through the Lense of an NGO" [3] paper, then put yourself in their shoes. Think about the IT resources a small NGO with 30-50 employees has. Maybe they have a sysadmin and a helpdesk guy. They are dead meat. The threats are so vast, spear phishing, target malware via MITM attacks, etc. It almost seems hopeless. But it is not just the NSA at the top of the heap, you have lots of foreign governments, which have direct access to your playing field via the internet.

Think about the resources that Google, Facebook, and Apple throw at security, then you see something like Operation Aurora [4, 5]. What chance does an ISP or small business have? None. Personally, it just seems like the entire model is broken. Yet, nothing seems to change, in that we are all just waiting for the next zero day to drop, and the cycle continues. All it takes is one targeted zero day addressed to a normal employee, the attackers gain access to the network, then move laterally [6, 7]. The odds are further stacked, in that you have a top tear team against a targeted employee, who doesn't even know the game.

ps. sorry for the tone of this

[1] [https://firstlook.org/theintercept/2014/03/20/inside-nsa- sec...](https://firstlook.org/theintercept/2014/03/20/inside-nsa-secret- efforts-hunt-hack-system-administrators/)

[2] [http://www.theguardian.com/world/2014/aug/13/snowden-nsa- syr...](http://www.theguardian.com/world/2014/aug/13/snowden-nsa-syria- internet-outage-civil-war)

[3] [http://www.mpi-sws.org/~stevens/pubs/sec14.pdf](http://www.mpi- sws.org/~stevens/pubs/sec14.pdf)

[4] [http://www.wired.com/2010/01/google-hack- attack/](http://www.wired.com/2010/01/google-hack-attack/)

[5] [http://en.wikipedia.org/wiki/Operation_Aurora](http://en.wikipedia.org/wiki/Operation_Aurora)

[6] [http://g0s.org/wp- content/uploads/2013/downloads/Inside_Repo...](http://g0s.org/wp- content/uploads/2013/downloads/Inside_Report_by_Infosec_Consortium.pdf)

[7] [http://intelreport.mandiant.com/Mandiant_APT1_Report.pdf](http://intelreport.mandiant.com/Mandiant_APT1_Report.pdf)

<comment><USER18> IMHO it can be done, using UNIX-based operating systems and standard open source software, if the infrastructure designer takes security _seriously_ and is committed and knowledgeable.

It's NOT the system administrator's work to _secure the network_. The ISP/minor IT company should have a security engineer to overview the network, although some sys-admins are extremely skilled when it comes to security.

There are so many security layers that can be implemented on a Linux/BSD server that makes the machine virtually un-hackable and IF anyone enters, all bells and whistles could start cheering.

Examples: GRsecurity[1], IPTables[2], Snort[3], chroots (or jails), VPNs, malware scanners (clamav, spamassassin), encryption and what-not.

To me securing linux desktops, especially simple ones (e.g. window manager + basic programs... almost like thin clients) is easy. Securing Windows XP/7/8/etc is extremely tricky BUT can be done.

Once you do all that, I'm 100% that you're going to be one hell of a target for anyone. And you really don't trust your team you can always hire people to test your network's security and improve it.

The most important thing though is having a strict general user policy: What users can and can not do must be crystal clear with no exception. When a 'tiger team' finds a _secure network_ , they usually target the people not the infrastructure.

[1] GRsecurity: [https://grsecurity.net/](https://grsecurity.net/)

[2] [http://www.netfilter.org/projects/iptables/](http://www.netfilter.org/projects/iptables/)

[3] [https://www.snort.org/](https://www.snort.org/)

<comment><USER19> How would anything you recommend protect you against an attacker who could (for example) have 3G transmitting keyloggers installed in your last shipment of laptops?

<comment><USER18> You choose your hardware carefully and if paranoid enough, you perform statefull packet inspection on your OpenBSD transparent router to know possibly what's the reason/content for every connection. Once you tag those that are standard, you start narrowing things. Again policy is what matters, if you allow torrents you are making your job extremely difficult.

I know easier said than done. Writing firewall rules/ confug reporting tools for every computer in the internal part of a network is hard too, that's why almost no one does it.

<comment><USER19> 3G keyloggers. Data egress is completely bypassing your network. (There are pwnie express boxes that include a 3G data link for bypassing target networks when sending results back.) I guess you could add physical scans for unauthorized radio transmitters to your security routine. Opsec is hard.

<comment><USER18> Sure, but that's a physical attack: You need someone to install the 3G Keylogger to your machinery.

I was talking on a keyboard-only level, but even that can be largely mitigate with proper policies IMHO.

That said, an insider is an achilles heel for every security scheme out there (e.g. Snowden).

</comment></comment></comment></comment></comment></comment><comment><USER20> How come Stellar PCS didn't check out the NSA documents that were made public (I assume the documents the journalist is showing them in the video are those public PRISM/snowden/TreasureMap docs) for any hints their operations were compromised ?

Just found this [https://firstlook.org/theintercept/2014/09/14/nsa- stellar/](https://firstlook.org/theintercept/2014/09/14/nsa-stellar/) which includes more narrative and GCHQ's involvement.

<comment><USER3> How come so many people are just ignoring all the Snowden files altogether? How come so little people use encryption, after more than a year that we have been told "the good news is, encryption works".

Please advocate for change, actively, with the people you interact. GPG, OTR, TextSecure, Redphone, Signal, decentralized services… If they are complicated, set them up yourself. And please, urge your representatives to act!

<comment><USER21> Unfortunately, some people still don't believe the scope or that the "good guys" would violate the constitution like this ("Just-World Hypothesis"). Others are still using various permutations of the "If you're not doing anything wrong..." nonsense.

And at least a few... are _collaborators_.

As PHK cautioned, the NSA/GCHQ/etc can submit patches or comment in development discussions just like everybody else, and at least some of the suggestions against using proper crypto are intended to keep the internet in plaintext.

</comment></comment></comment><comment><USER22> If you're a radical, islamist, leftist, etc. then _maybe_ you're a target for NSA, GCHQ.

if you're a sysadmin at a telco or infrastructure provider your _definitely_ a target for the NSA, GCHQ.

let that sink in.

infrastructure these days also means AWS, facebook, youtube, twitter. every piece, site, offer that might be used by ISIS, for example.

</comment><comment><USER23> I'd be really interested to know what they did to get the access. Did this guy have malware installed on his machine? Do we all have Malware installed on our machines? Is there any way to protect yourself from an adversary as powerful and competent as the NSA?

<comment><USER24> The only real proof of "access" I saw, was a single customer's username and password. That could have been obtained through guessing, or compromising that customer. So it seems like cautious and competent engineers aren't necessarily all compromised. I'm open to more evidence though.

The other things seemed to be network topology, IP addresses, and engineer lists, which are fairly public.

<comment><USER23> His name is on NSA slides, right? He's either working for them or they have his private key. Really how difficult would it be for the NSA to get that if they wanted it?

I have no idea but I'm able to believe easily.

<comment><USER24> Like some others here, I wondered if "task" could mean to recruit.

I basically agree NSA could pwn anybody they want, but there are probably other considerations such as how obvious they want to be, whether the target is valuable enough to reveal zero-days nobody else has seen yet, etc. Maybe it's wishful thinking, but I'd like to believe if you don't do things like open unsolicited email attachments, you're still pretty safe.

But, perhaps as the lead engineer of an ISP "interesting" people use, nothing is off the table and he has been pwned repeatedly.

<comment><USER25> I remember reading about one method where they served up their own versions of Facebook when requested from a target from compromised hosts that are near that target as a way to collect credentials.

I forget the name of that method, but according to the documents it was used to target sysadmins. Though if you use a password manage with unique passwords for every service that should help protect you.

For more details: [https://firstlook.org/theintercept/document/2014/03/20/hunt-...](https://firstlook.org/theintercept/document/2014/03/20/hunt- sys-admins/)

</comment></comment></comment></comment></comment><comment><USER26> If I were caught hacking into a private network of computers without authorization even if I had a "good" reason to do so I would be breaking the law and throne in jail. So why is the NSA allowed to do the same?

<comment><USER27> Within the US, the NSA operates with legal impunity. It is beyond the reach of the law. At least so long as it's acting under official mandate -- rogue agents are apparently sanctioned if pursuing information for personal reasons. If caught.

Outside the US, NSA operates with the diplomatic and military support of the US. It's not a matter of an individual hacker, but "an international incident" should something arise. There's _some_ risk that an agent or operative (non- agency employee acting on behalf of the NSA) could be caught, but that would vary by field of operations and relations between that country and the US.

<comment><USER28> And for such cases as in NSA, where the actions, though technically illegal, were clearly intended by the leadership - "sanction" can easily mean an official written reprimand stating "this is not acceptable", followed by a cash bonus and a promotion.

</comment></comment></comment><comment><USER29> that feel when ~5:15 and the username/password to an account with "deep access to the network" is horizon/h0r1z0n

<comment><USER24> I think it was deep access to one customer's network, and probably their own fault for choosing lame credentials.

</comment></comment><comment><USER30> Any non-flash version of this video?

<comment><USER3> You can get it in H246 from Vimeo:

[http://vimeo.com/106026217](http://vimeo.com/106026217)

<comment><USER30> Awesome, thank you!

</comment></comment></comment><comment><USER27> Is there an alternate source for the video? I cannot get it to play at all under Linux / Chrome.

<comment><USER12> Here it is on Vimeo:

[http://vimeo.com/106026217](http://vimeo.com/106026217)

Worked fine for me in FF on linux. Might be worth a try in Chrome.

<comment><USER27> Vimeo _does_ work for me generally. I can also download it with youtube-dl.

</comment></comment></comment><comment><USER31> There's this great line in Dataclysm (written by the guy who wrote the OkCupid blog) about how the NSA recruited from the best math students at Harvard. "The people spying on us are extremely, extremely smart."

<comment><USER32> Don't we generally assume that the NSA hires math guys for math purposes, not surveillance purposes?

<comment><USER33> What do you think they use the math for?

<comment><USER32> My point is that mathematicians are not necessarily or even probably a good fit for intelligence analysis ("spying"). I wouldn't describe the stuff NSA mathematicians ostensibly get up to as spying. See this for example: [https://www.nsa.gov/careers/career_fields/mathematics.shtml](https://www.nsa.gov/careers/career_fields/mathematics.shtml)

</comment></comment></comment></comment><comment><USER34> Reminds me of the Google engineer response:

[https://plus.google.com/+BrandonDowney/posts/SfYy8xbDWGG](https://plus.google.com/+BrandonDowney/posts/SfYy8xbDWGG)

</comment><comment><USER35> What about 2048 RSA public key cryptography performed on the application level? Didn't the Snowden leak say the NSA still can't crack 2048?

</comment><comment><USER36> This also illustrates the weakness of using just a password for authentication to anything of value.

</comment><comment><USER37> _Requires Flash to watch_ :\

<comment><USER3> Can be done with H264: [https://news.ycombinator.com/item?id=8316396](https://news.ycombinator.com/item?id=8316396)

</comment></comment><comment><USER38> This crashes my shockwave flash plugin - should I start to be paranoid?

</comment><comment><USER39> They must've known?

</comment><comment><USER40> This is the most intense video I ever seen since the Snowden revelations. I could almost feel Ali's feeling of complete violation. This is absolutely chilling material.

<comment><USER41> To see a fellow german talk in my native tongue about learning that he himself was targeted made this so much more real. We europeans really can't trust the US anymore. It's so sad. I really liked the idea of that place, but who hacks their friends?

</comment></comment><comment><USER42> Enjoyable. A rare opportunity to witness an expression of someone who got p0wned well beyond his level of comprehension. Look how he strokes his pen in disbelief, that poor German dude. "I know those switches", Mein Gott.

As of today, there are two kinds of people in the world: those who believe we're still stuck in post-9/11, and those who realised we are now in post- Snowden.

There is a third kind who have Facebook accounts, but those are just the nature's way of saying that Darwin got it right.

<comment><USER43> _There is a third kind who still have Facebook accounts, but those are just the nature 's way of saying that Darwin got it right._

That's a non sequitur.

Having a facebook account or not is an almost negligible factor in the grand scheme of things.

The kind of information commonly exposed on facebook can be more conclusively inferred from other information sources.

Do you use a smartphone? Skype? Any search engine?

Does your home internet come out of a plastic router? Do you click on the little lock icon every time you go to a SSL site, to verify the certificate hasn't changed? Do you know the fingerprint of the legitimate facebook SSL certificate?

Think about what any one of the above devices "knows" about you in comparison to what facebook knows about you.

<comment><USER42> True, true.

The only difference is that somehow Google still maintains a straight face telling people they are not guinea pigs in their next study. Zuck never had that option in the first place, their only strategy is to maintain grip on the population at all costs, and no means are too sleazy.

What you want to try is to shut down your Facebook account and check some e-mails they will be sending you for months to come... Prepare for the drama. They will be showing you the cutest photos of your family and closest friends, saying they're all devastated because you've gone antisocial.

Zuck broke our hearts.

<comment><USER44> Hm, I shut down my account, and I haven't gotten any emails from them... perhaps they've changed?

<comment><USER42> No, I doubt they have. What's more likely is that you took an option to to opt-out from any further communication from Facebook, something they reluctantly offered when all attempts to connect with you emotionally failed.</comment></comment></comment></comment></comment>


Why am I being endorsed for skills and expertise I do not claim on my profile? - <AUTHOR> http://community.linkedin.com/questions/16624/why-am-i-being-endorsed-for-skills-and-expertise-i.html?page=1&pageSize=10&sort=votes <comment><USER2> I quickly realized how little value the endorsement system in linkedin was when my uncle endorsed me for Groovy. In my mind I like to think he was picking it as a personality trait and not a skillset.

<comment><USER3> > my uncle endorsed me for Groovy (...) as a personality trait

I laughed harder than usual at that. I wonder what a "groovy" professional would be? :)

<comment><USER4> When I read that I imagined LinkedIn is refering to the groovy programming language [http://groovy.codehaus.org/](http://groovy.codehaus.org/)

Of course... that doesn't make a lot of sense for a personality trait either... It makes as much sense as describing someone as C++. Maybe I'm misunderstanding something here.

<comment><USER5> Groovy (or, less common, "Groovie" or "Groovey") is a slang colloquialism popular during the 1960s and 1970s. It is roughly synonymous with words such as "cool", "excellent", "fashionable", or "amazing", depending on context.

[http://en.wikipedia.org/wiki/Groovy](http://en.wikipedia.org/wiki/Groovy)

<comment><USER6> That this needs to be explained makes me feel old.

<comment><USER7> I suspect that it is a geographical-cultural thing, not an age-culture thing.

edit: for clarity

<comment><USER8> Old people have different cultures.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> Interestingly, "Murder" is a legitimate skill on LinkedIn. Not practitioners, but criminal lawyers advertising their expertise.

[http://www.linkedin.com/skills/skill/Murder](http://www.linkedin.com/skills/skill/Murder)

The related skills sidebar is also hilarious.

<comment><USER10> I'll tell that to my brother-in-law. He writes thrillers featuring a "corporate trouble shooter" specialized in "fortunate accidents" as protagonist, and he pretends to be his main character on social networks. This would be just perfect for him.

</comment></comment><comment><USER11> I don't think endorsements are meant to be true endorsements. If they were, LinkedIn would have written a weighted endorsement system by now (where an endorsement from someone with expertise in some field means more than, say, an endorsement from your realtor about your JavaScript skills).

It's just crowdsourced tagging with a different name. This way, recruiters who pay LinkedIn $10k/yr can more easily search profiles for keywords.

<comment><USER12> I think this is absolutely right on. But a smart recruiter would realize that some measurable chunk of endorsements are selected by people who have no clue as to your skill set or level of ability in that particular area -- people who, for example, logged in, saw four possible endorsements they could give people in their network at the top of their screen, were feeling beneficent that day, and clicked on all of them. Or someone who you just endorsed for a skill, and they see that you did, and want to endorse you for something but already endorsed you for the 10 skills you preloaded, and so they just pick the first thing LinkedIn recommends.

Presumably LinkedIn has some algorithm by which they look at people with similar skill set groupings, and say ah, statistically most of the people with this set have these three additional skills, and then farm that out to your network to see who endorses it. New endorsements tweak the skill set, which feeds back into the algorithm. It's almost like a glorified, twisted game of _The Game of Life_.

<comment><USER13> _a smart recruiter would realize..._

Nobody said the smart recruiters are their target market.

<comment><USER14> Sounds like a narrow niche to me.

</comment></comment></comment></comment><comment><USER15> I'd think this is pretty straightforward.

The real business of LinkedIn is selling services to employers and space to advertisers [0].

The more data points they have on each user, the more numerous results they can return to recruiters and more specific targeting they can do with ads.

Endorsements are a way to easily, indirectly expand a user profile by turning the additions into 1-click suggestions and consents rather than requiring real action by the user.

If it were socially acceptable I expect they'd ask users to upload profile photos and estimate salaries for their connections too.

0: [http://www.forbes.com/sites/tomiogeron/2013/08/01/linkedin-q...](http://www.forbes.com/sites/tomiogeron/2013/08/01/linkedin-q2-earnings- revenue-beat-street/)

<comment><USER16> > The more data points they have on each user, the more numerous results they > can return to recruiters and more specific targeting they can do with ads.

The thing is that endorsements at least are pretty noisy data. What's the point of shooting for quantity at the expense of quality?

<comment><USER15> _> "The thing is that endorsements at least are pretty noisy data. What's the point of shooting for quantity at the expense of quality?"_

Personally, the endorsements I receive are fairly noisy, but I'm acting as the second level of informed filter by choosing what to accept.

The endorsements which make it to my profile are accurate.

Unless I'm unique, I'd expect the resulting graph of accepted endorsements to be reasonably useful.

<comment><USER16> Well the classic example of garbage data is all recruiters getting endorsed for the two dozen technologies they are headhunting for.

</comment></comment></comment></comment><comment><USER17> I have a similar problem: I am endorsed for skills that I do claim on my profile by people whose endorsement I do not want. You can remove individual endorsements without removing the skill itself. Here are the instructions a friend of mine found for this:

Login to LinkedIn; Hover over "Profile" and click on "Edit Profile"; Scroll down to "Skills & Expertise"; Click on "Edit"; Click on "Manage Endorsements". It is not highlighted but it will accept a click when you hover.; Click on the skill where you want to remove endorsers and then uncheck any endorser you want to remove.; When you're done, click on "Save";

<comment><USER18> Following the same procedure, you can also choose not to display your endorsements in one shot: rather than click "Manage Endorsements" instead click the down-arrow next to "Display your endorsements" and choose "No, do not show my endorsements."

</comment></comment><comment><USER19> One of my friends had everyone endorse him for "knitting" and "problem gambling"... The fact that these are even options is laughable.

<comment><USER20> My friends colluded to endorse me as 'Mexican.' I'm not Mexican.

<comment><USER21> Mexican ... ? I don't even ... only idea I have is that it's related to food.

</comment></comment></comment><comment><USER22> We actually had an "endorsement war" between my coworkers last week.

Some of the results: Memes, Poetry, Salads (ten endorsements!), Tree Identification, Punk, Pickles (not to be confused with Cucumber), Shampoo, Extortion, Glitter Tattoos.

<comment><USER23> Murder

</comment></comment><comment><USER24> I recently just bailed on Linkedin. I was getting zero utility from it, and didn't see any possible future where I would. I ache to pull the trigger on Facebook as well, but the longer I wait the more entrenched it becomes for my family.

EDIT: removed superfluous "the".

<comment><USER25> I received a phone call at work from someone trying to sell JRebel, after all the recruiters that call through to the office(doesn't look good) I had to hide my linkedin profile.

</comment></comment><comment><USER26> A former coworker of mine endorsed me for, among things I actually do know, "lubrication", "flexibility" and "potatoes".

It took me a good while to notice and another good while to stop laughing. Of course a tiny bit less observation on my part could have published that to my actual profile.

</comment><comment><USER27> I was (friendly) trolling a colleague who's very entrenched into Java World. The troll started as a genuine attempt to suggest a new tech he'd like; Scala. Turned out he wasn't interested at all, but the gag kept rolling about him learning Scala.

I got the idea that I could get a bunch of people to go and endorse him for Scala on LinkedIn. The (far fetched) goal being that he'd inadvertently accept the endorsement and end up returning in the top results for local Scala experts.

So I went and endorsed him for that. He didn't fall for it. However I also endorsed other colleagues. One of them caught on the troll and endorsed me back for 'Genocide' and a bunch of atrocities.

Maybe people are just trolls like I am and endorse OP for the sake of lulz.

</comment><comment><USER28> The whole endorsement scam completely destroyed my trust in LinkedIn. I find that they are out soliciting endorsements on my behalf from my professional network _without my knowledge_ to be a heinous misrepresentation and a horrible breach of trust. I haven't logged in there in more than a year.

</comment><comment><USER29> I endorse anyone who mentions these silly things to me for "Natural Horsemanship".

<comment><USER30> My favorite was the paired endorsements "Free" "Colonoscopy".

</comment></comment><comment><USER31> New hobby: searching LinkedIn for developers that work at LinkedIn and endorsing them for crazy stuff.

<comment><USER32> Damn, and I JUST deleted my account this morning.

</comment></comment><comment><USER33> I turned off the ability for endorsements to automatically appear on my LinkedIn profile. Instead, I highlight the skills I want to highlight.

The fact is your endorsements are going to trail your most recent skills by years in most cases, and so unless you've been doing the same thing year in and year out, endorsements present a distorted picture of your overall skill set.

This is especially true in technology, where your most recent skills are generally your most valuable ones.

<comment><USER34> How do you turn it off?

<comment><USER35> You disable endorsements from appearing on your profile: [http://help.linkedin.com/app/answers/detail/a_id/33198](http://help.linkedin.com/app/answers/detail/a_id/33198)

What you can't do is prevent getting endorsements in the first place. LinkedIn will not let you disable receiving endorsements.

<comment> alok-g Could someone confirm that this will still keep my existing endorsements just in case I want to show them again later?

<comment><USER36> Yes your endorsements are always there. You will continue receiving endorsements too. They just won't be displayed if you "disable" them.

<comment> alok-g Thanks.

</comment></comment></comment></comment></comment></comment><comment><USER37> Before I became a developer I was a banquet manager and I've had numerous colleagues from my former career endorse me for software skills they can't possibly know I have. Though I do get a chuckle when one endorses me for 'Java' as I always imagine them thinking "yeah, he knew a lot about coffee"

As presently implemented the endorsement system is rather pointless, but I do think there's some potential for improvement in making it more meaningful. Perhaps make it more like the recommendation system where you'd be required to actually write something instead of just clicking a button.

</comment><comment><USER38> They are the Facebook "Like" button for your career, and equally valuable.

</comment><comment><USER39> I think the problem is that LinkedIn makes it too easy and the incentives are out of whack. LinkedIn figures out what you may be good at, and then presents it as a button to all your contacts. Brain dead simple. That's fine, but the problem is that people are incentivized to endorse you, either because they may think it's something nice they can do for you (and doesn't hurt them and takes no effort), or because they may be looking for reciprocity. It's just a terrible system.

</comment><comment><USER40> I joined Linked In 3 months ago. Personally I did it as I was starting a new company and wanted to "discover" this magical sales and marketing tool I've been disappointed by the noise, and general low level of barrier to entry on most things. The fact that endorsements are so easy, or that making a good profile, filled with content look attractive decreases the value of it as a tool. I dislike Linked In, I wish I hadn't joined it.

<comment><USER41> It's gotten pretty bad now but it wasn't always that way. I think I've had an account for 10? years. It used to be the professional Facebook, so everyone joined. It wasn't as spammy as it was now and I definitely used it to keep in touch with people that I used to work with and to research jobs. I guess they are trying to be the actual Facebook now... I really only check it once every month or two in order to delete invites from recruiters. It's not a social destination, but it seems like they want people to log in every day. Bleah.

I already maintain a personal and a work facebook account (for example to add someone as a facebook app developer on an app you own you have to be friends with them, but I don't want to be "friends" with all my co-workers). There's not much you can do about linked in at this point since it's ubiquitous. I just try to avoid the spammy bits.

</comment></comment><comment><USER42> I deliberately, and almost ruthlessly enough, limit my LinkedIn connections to people I've worked with closely enough to evaluate each other. I really want it to be a career resource. The set of endorsements under the system as currently implemented, bears little correspondence with what those people would say about me if you asked them. This makes the data, and LinkedIn itself, less useful.

<comment><USER33> I recently removed fully 1/3 of my connections. They were older connections that didn't have much bearing on my career as it exists today.

</comment></comment><comment><USER43> Cached/mirror: [http://webcache.googleusercontent.com/search?q=cache:ql0v0ab...](http://webcache.googleusercontent.com/search?q=cache:ql0v0ab3FmkJ:community.linkedin.com/questions/16624/why- am-i-being-endorsed-for-skills-and-expertise-i.html+&cd=1&hl=en&ct=clnk&gl=us)

</comment><comment> alok-g Here is what I would guess LinkedIn must have been thinking: Nearly everybody boasts about their achievements in their profiles and CVs, making it hard for readers to tell high performers apart from low performers. So what could be a better way than to have others vouch for people </ironical>? This is very similar to older days of websites trying to enhance their rankings by hook or crook, and Google measuring a website's relevance by how many other websites link to it (ultimately leading to the PageRank algorithm).

Unfortunately, this has not worked the way they may have intended. Any widespread data-oriented company has to work hard to keep noise at low levels. A means probably intended to reduce the noise in this case has become a source of noise rather.

<comment><USER44> missing opening tag on line 0 +1 parsing skill

</comment></comment><comment><USER45> From the linked page comments: "Marla Collins Jun 21 at 04:50 PM I personally think this is an intentional glitch LI has implemented to try and assimilate a face book type of social interaction. _I can tell you that the people who have allegedly endorsed me have done nothing of the kind any more than I have gone to thier profile and endorsed them after years of not really interacting with them on LI._ This is a violation of our trust and is turning LI into just another annoying social networking site. "

Emphasis mine. If this is true, this is even worse than the spammy "would you like to endorse?" stuff.

</comment><comment><USER46> I got one endorsement for Ruby (I have it in my profile that I am learning) despite not listing it as a skill, and I'm now occasionally getting messages from recruiters.

</comment><comment><USER47> If you're relying on linkedin to actually get a good job (except maybe for people just getting there first entry level job) then you need all the help you can get...

</comment><comment><USER48> The truth is, Does it really matter. People look at your skills or skills people think you're good at, and it allows you more breadth and job opportunities.

</comment><comment><USER49> People love to hate the endorsement system, but I have found that while I get endorsements from people who clearly aren't qualified to asses my skills, it still ends up such that my most-endorsed skills are actually fairly close to reality. As a recruiter you'd be foolish to use this at face value, but for doing a rough search to narrow down candidates, I don't think it's way off.

<comment><USER50> Due to the way LinkedIn asks people about endorsing others, endorsers are usually people who are very kind to you, like your mom, your close friends, relatives.

<comment><USER49> Sure, there's a lot of noise in there. What I mean is that given enough endorsements, you kind of tend to get most endorsements in the skills you actually work with. It does not say much about how well you perform in those, but for searching out potential candidates, I imagine that it's fairly accurate.

</comment></comment></comment><comment><USER51> "Girls only want boyfriends who have great skills." \- Napoleon Dynamite

<comment><USER52> Please please please someone endorse me for "nunchuck skills"

</comment></comment><comment><USER53> Linked-in is a bit like craigslist in that it has huge network effects, lots of flaws and is hard to attack.

Still, I believe that there is a window of opportunity for a determined group to disrupt linked in in much the same way that Google attacked Altavista, by offering substantially better quality combined with a different business model.

</comment><comment><USER54> having spent some time during linkedin boom in a company that separates their developers as frontend vs backend (which is moronic), i'm now labeled as frontend just because that was what i was doing during said boom.

gladly my linkedin profile is as useless as my gplus one (which i have to have to comment on youtube kitten videos)

</comment><comment><USER55> Linkedin sent an invite to every person in my gmail account, hundreds of people, and was then spammed with pages of people accepting my friend request. I had no intention to do this, it was from a button that said "sign in with your gmail" that I must have clicked because I thought I wasn't signed in.

</comment><comment><USER56> I think it's great! I've got a dozen recommendations for PPC if I ever decide to learn it.

Alright, the system's a joke.

</comment><comment><USER57> How to fix endorsements - rather than asking if someone has a skill, take two people in your circle claiming the same skill and ask who's more competent at that. Repeat this for all people in your circle with that skill and you can bubble sort people into who's best in your eyes. Next take those people's standing within their respective circles, perhaps weighted by some measure (e.g. people working in the same field's opinion counts more) and you start to get a good idea of how people compare to one another for the skills you're interested in. Even if those people don't have anyone in common you can use the 6 degrees of separation to use their connections' connections to approximate their standing compared to one another.

<comment><USER57> (of course, this assumes that you wish endorsements to have meaning; if you're just using it to increase user engagement with your site the current method works fine - until people get fed up with putting in redundant effort).

</comment></comment><comment><USER58> I've been actually trying to upgrade my skills as a software developer. I am currently a level 5 Wizard.

[http://www.linkedin.com/pub/justin- nash/26/7aa/163](http://www.linkedin.com/pub/justin-nash/26/7aa/163)

</comment><comment><USER59> I just added "King of England" to my skills... I'm waiting to see how long it takes somebody to endorse me for that.

In hindsight, I should have added something like "TARDIS Repair" or "n-dimensional Temporal Hypergeography". :-)

</comment><comment><USER60> I am being endorsed for technologies that people in other departments think I use (which I've often not touched), while corporate management are busy endorsing each other for "Entrepreneurship." :)

My most endorsed skills I have not used for years.

</comment><comment><USER61> Does John Resig know about jQuery?

[https://twitter.com/notasausage/status/360788936282501120/ph...](https://twitter.com/notasausage/status/360788936282501120/photo/1)

<comment><USER62> Is DHH interested in Rails opportunities?

[https://gist.github.com/dhh/1285068](https://gist.github.com/dhh/1285068)

</comment></comment><comment><USER63> Push-button endorsement systems are meaningless without a corresponding downvote mechanism. There have to be both sides of the coin, or else you get nothing but meaningless and empty results.

Of course, if you had more of a "karma" style system, you'd have to implement a way to mitigate abuses of the downvote mechanism. But I feel like that's an easier task than trying to derive meaning from endorsements that cost the endorser nothing, and hence mean nothing.

If the stakes aren't real, the results aren't real.

<comment><USER64> > Push-button endorsement systems are meaningless without a corresponding > downvote mechanism. There have to be both sides of the coin, or else you get > nothing but meaningless and empty results.

Those are not the only techniques available. If you look at how Google's PageRank works, a link is like an endorsement, but there's no downvoting possible.

<comment><USER63> Good point. There's probably a better way to do it than downvotes (also, now that I think about it, who's ever going to downvote someone else on LinkedIn?).

</comment></comment></comment><comment><USER65> I'm primarily "endorsed" for HTML5 and jQuery. While I'm good at those, they're neither impressive to list, nor are they my core competency.

So I consider the whole thing a joke and endorse people for the most ridiculous things I can. Here are some of my favorites: footprints, sandwiches, trees, safety, anger, hunger, eating, money, typing, girls night, walking, Goal-oriented individual with strong leadership capabilities, MS-DOS.

</comment><comment><USER66> In my area linkedin is quite important. However, I have been frustrated with their service for a while now.

One thing I really hate is that every change to my profile propagates to my whole network and to the headhunters. I am ok with spamming the headhunters, but I feel very weird having my friends receiving updates like this from me. Sometimes you are just keeping a profile updated, you are not getting promotions or something worth noting.

<comment><USER67> You can turn off broadcast notifications in your settings.

</comment></comment><comment><USER58> In all seriousness though, I think it's a UI change they did. It didn't seem to be this bad until lately. I've noticed whenever you login it asks you with a one click button if you want to endorse these people for these skills, _randomly_.

I think their goal was to create more 'value' in their product but in fact made it less reliable and therefore less 'valuable'.

</comment><comment><USER46> For some reason, everyone endorses me for SEO, when that is the really the only type of online marketing I've never done.

<comment><USER68> That is the incidental value in the system. You get to see what people think you do for a living.

</comment></comment><comment><USER69> If you were to read my profile and see my endorsements, you would think I am the master of all that is, ever was and ever will be in software configuration management using IBM ClearCase. I half-jokingly thought that my former workplace mandated everyone to endorse someone they worked with in the past for this particular skill.

</comment><comment> alok-g A related issue I have is that they limit the number of skills to 50. What are included in "skills" these days are sometimes experience with very specific tools here and there. For a multidisciplinary person like me, I am tired of prioritizing what to keep and what not to within this limit of 50.

</comment><comment><USER70> LinkedIn suggested that one of my friends should be endorsed for 'pregnancy'. Well, I couldn't resist that suggestion so I went for it.

I called him up the next day and we both laughed about it. He then added it to his profile. Since then he's been seconded on that skill by somebody else, too.

</comment><comment><USER71> What they ought to do is to give a different notation for being endorsed by someone else with the same skill.

</comment><comment><USER72> I would very much like for someone to explain to me what the point of LinkedIn really is. I just don't get it. Aside from your colloquial spamming, stupid recommendations and business "contacts", I really fail to see who benefits from that platform aside from the investors.

</comment><comment><USER73> I remember I once setup a js script which ran every 5 seconds and clicked the 'endorse-all' button. In this way within 30-45 minutes I endorsed many of my friends of skills which they don't even know.

Was good return for me as many returned back favor thinking I spent time for them.

</comment><comment><USER74> Perhaps LinkedIn will get the message when everyone endorses each other for King of France.

</comment><comment><USER75> I've noticed that people tend to endorse each other for not-so-cutting-edge technologies as a joke.

"You have endorsed Coworker for SVN, Java and XSLT"

Where I work, I don't think a week has gone by without this happening. It's all in good fun. :)

</comment><comment><USER76> I got endorsed for Javascript by one of my undergrad professors that teaching business communication...among other equally as WTF endorsements.

I always sorta thought that LI was randomly generating these things somehow.

</comment><comment><USER77> Can I just delete my LinkedIn account without performing professional suicide? It's probably out of date, and I use it for less than 5% of what's actually going on there.

</comment><comment><USER78> I have a friend who endorses people for "Bikini Waxing".

</comment><comment><USER79> I find it odd when people I don't know but I must've hit accept on their LinkedIn request at some point endorse me for stuff. Smh

</comment><comment><USER80> One of the most awesome skills I regularly get ensorsed for is "general awesomeness" I believe there is not even a filter on it

</comment><comment><USER81> I wrote a chrome extension for linkedin that (among other things) deletes endorsements from the dom of any profile page

</comment><comment><USER82> That's a good idea, linkedin! I should endorse few of my competitors for midnight poll dancing or something ...

<comment><USER83> Using A/B testing to strip away bad requirements?

</comment></comment><comment><USER84> People endorse me for stuff they don't really know that I'm good at... I find that oddly uncomfortable.

</comment><comment><USER85> whenever i received an endorsement for a certain skill from someone who has never seen me perform that skill i quickly remove it. recently however, after trying to remove endorsements several times, they still appear on my profile. i've since given up on linkedin for not providing any integrity.

<comment><USER86> I've never updated my skill set on LinkedIn. Don't see any value to it. And yet some of my friends who have no idea what I'm working on endorsed me for random programming skill set. There should be a credibility factor when someone endorses you. Say the person endorsing you for skill set x should have been endorsed for that skill set x at least n times before being judged credible. Right now, it's just a "you scratch my back, I'll scratch yours" function.

</comment></comment><comment><USER87> I just realized I've been recently endorsed for "fly fishing" haha. That system is quite a joke.

</comment><comment><USER88> I tried to endorse my friend for Defense Against the Dark Arts. Didn't work. Useless I tell you.

</comment><comment><USER89> Yes! I have people who I've never worked with endorse me. The endorsement thing should stop.

</comment><comment><USER90> I've gotten four endorsements for 'Butter', none for 'Balls' so far.

</comment><comment><USER91> Somebody please endorse me with the "Dismissive of Linkedin Endorsements" skill...

</comment><comment><USER92> My brother-in-law (an insurance salesman) endorsed me for XML. There'ya'go.

</comment><comment><USER93> linkedin guys are either spammers or they just don't know how to build the "email notification preference" thingy. I suspect they keep this feature broken by design...

</comment><comment><USER94> i reckon linked in should have mandatory comments on an endorsement to explain why or what experience that endorser had regarding your skill.

</comment><comment><USER95> linked in got really weird when it started suggesting to people to endorse people.

</comment><comment><USER96> Good idea executed terribly.

</comment><comment><USER97> Linkedin sucks.

</comment><comment><USER98> LinkedIn has become effectively useless as an actual talent/hiring tool, at least in my area, though it remains a pretty decent contacts tool.

The recommendation system is just as fictitious as the endoresements: it has been my experience that many recommendations are quid pro quo and have no legitimacy. Many come in a circular fashion so when assessing talent you need to walk through each of the players trying to figure out what their real motivation was.

I've watched the sausage being made. I remember once when we had to choose a team member to punt at one shop I worked, and there was universal agreement on the target, the team long having grievances about their productivity, poor quality product, etc. The deed gets done, and over the next week I see a several of my peers, including his highly critical direct manager (who hands out absurd recommendations like candy), author lengthy, lauding recommendations for this person. They were trying to trade in a recommendation to assuage their guilt, or for future considerations in return. This seemed incredible to me because it should be professionally damaging to do something like that, but in the real world it simply isn't: There are no consequences or downsides to handing out ridiculous endorsements and fictional recommendations.</comment>


Distributions vs. Releases: Why Python Packaging Is Hard - <AUTHOR> https://pydist.com/blog/distributions-vs-releases <comment><USER2> Most of my pain with python packaging comes from incompatible changes in the toolchain.

Some years ago, pip started distrusting HTTP mirrors, and while you could add some options to force it use HTTP, these options weren't present in previous pip versions. Which meant that you now had to provide options depending on the pip version -- which is harder if you don't call pip directly (for example through dh-virtaualenv).

We switched to HTTPS, but one with a TLS cert signed by a company-internal CA. Getting pip to trust another CA is non-obvious, and again depends on the pip version. So another version-dependent treatment necessary.

Then pip suddenly moved all of its implementation from pip.something to pip._internal.something, stating that it was documented all along that you shouldn't import anything from the pip.* namespace. But, the package names all didn't start with an underscore, so, by python convention, looked like perfectly fine public classes and methods.

Moreover there simply isn't any public API to do things you _really_ don't want to reimplement yourself, like parsing `requirements.txt` files.

As soon as you want to do anything slightly non-standard with pip/distutils/setuptools, you find a solution pretty quickly, and that solution turns into a nightmare when you upgrade any of the components involved.

Also, finding a local pypi mirror that lets you inject your own packages _and_ deals with wheels correctly _and_ doesn't generally suck _and_ is somewhat easy to use... not easy.

<comment><USER3> Agreed; for something that managed to become the de-facto solution for Python package installation, pip always felt surprisingly unreliable to me.

My solution at a small company, where we couldn't waste time with this crap, was to vendor everything: when a developer updated the requirements.txt, they would also install the packages in a project directory (using "pip install -r requirements.txt -t ./dependencies") and commit that. That way, the rest of the process (CI, packaging, etc) could just checkout the repo, set PYTHONPATH=./dependencies and ignore pip completely.

</comment></comment><comment><USER4> Nix's usage of source hashes to pin every package is more and more prescient, as each language's custom-written package manager reinvents the wheel. (Pun indented!)

<comment><USER5> There are plenty of situations where the output of the build process is not deterministic on only the source.

<comment><USER6> Example?

<comment><USER5> Install Python from source with and without readline-devel in the ld library path.

Or any other piece of C software with macros that depend on the build environment. Same .c file in, different binary out.

<comment><USER7> This sort of thing is IMO exactly why containers have merit. Let the software vendor decide what to distribute or others jump in and fill the gap. Powerusers can still read the dockerfiles and decide what do do or even build themselves. When it's built the image is well defined, i.e. not depending on host environment, including files stem except volume mounts. When will we get a whole desktop distro like this?

<comment><USER8> Many of the pieces which go into a desktop have a strong assumption of being able to interact with other components in ways other than a network socket. Containerizing an application is one thing, but how do you containerize something that has plugins? What about desktop widgets? Do they go in their own containers, or get mixed in with the host's somehow? How is ABI compatibility enforced now that we're in container land and none of that matters any more?

What about client/server components where there's dbus in between, like the network manager or volume control?

These are solvable problems, but it's been enough of a challenge building a server OS from containers (RancherOS), and building a desktop OS that way is a significantly harder problem.

<comment><USER7> > containerize something that has plugins

I don't really see the problem there, I see the plugins as essentially just data (i.e. stored in a mounted volume), for which updating and versioning is in the domain of the application itself or maybe some standardized library it uses.

desktop widgets: essentially the same thing, it's a plugin to the desktop environment and can be stored as a volume mount on the DE container.

dbus is probably something that would require an evolution on the container side, or alternatively it would need to be all abstracted into network interfaces. another possible way to look at it is to have a layer between kernel and containerized userland that is responsible for manipulating all the physical host things in the traditional way, and the examples you give are exactly that. maybe this sort of thing should continue to be distributed tightly together with the kernel.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> I only got interested in Python again after I became aware of tools like Poetry [https://poetry.eustace.io/](https://poetry.eustace.io/)

<comment><USER10> Looks very interesting, but the part of the installation instructions where it says that its directory "will be in your $PATH environment variable" worries me – is the installer taking the liberty of messing with my configuration without asking first?

</comment></comment><comment><USER11> If you take this further, and can compile any of the "C" code (open source), then you can link that existing code as one single binary - e.g. "python" \- basically python with all compiled code needed, thus: \- You no longer need .dll, .so, etc. to go along, you can package the rest of the .py files into a .zip and slap it at the end of the now "fat" python binary, hence effectively having one executable - a bit of a "go" style releasing... \- This can even have smaller sizes, but careful (with FFI) - because only the code that is exposed to be used from Python would get linked in the fat "python" binary.. that is - with "FFI" is more tricky, as you have to force symbol to be kept \- You can obviously do that with other runtimes - java - e.g. your whole "java" can be down to one executable really

But to get there, you need sane build system, in order to explain these dependencies outside of python/java/whatever... And obviously the source code (although precompiled static libs should work in this scheme too)...

The goal again - just to have one thing, and most importantly no .so, .dll, .dylib lying around,

<comment><USER3> This is what tools like py2exe and PyInstaller do.

<comment><USER12> You can do this with plain zip files as Python can unpack and execute in- memory. But you have to extract non-Python code such as C libraries at runtime somewhere so the OS can link them. I would be interested to know how the packagers you mentioned get around this, or if they too use temporary files and unpack.

<comment><USER13> This is exactly how I package my python scripts. The answer to your question: Memfd [https://magisterquis.github.io/2018/03/31/in-memory-only- elf...](https://magisterquis.github.io/2018/03/31/in-memory-only-elf- execution.html)

And zipimport pip package

<comment><USER11> Right, there are OS-specific approaches on all systems, but if you are able to compile Python itself along all open source C/C++ libs that your .py-thon file might use, you end up with exactly one executable (you can possibly even go further packaging the .py files in some compressed (bytecode compiled) inside the same binary) - e.g. all you need is there, and the approach would fare better across more systems (e.g. does not rely on advanced techniques, no need to unpack the .DLL in temp folder, etc.). Also better for post-mortem debugging - your .PDB, and your .DWP (or whatever symbol files) are.. well just one single file, rather than several.

Same thing for java, dart, etc. This makes it much easier to deploy, use, download, as it's basically "zero"-config install, no need for scripts to tweak your PATH, etc.

(but it has limitations clearly, if you use proprietary code, or you can't compile given open source library bundled with you - e.g. LGPL code, that you have to dynamically link, then again if you are project is open source this is no longer that much of a limit (but would not know that for sure))

</comment></comment></comment></comment></comment><comment><USER14> Author here, happy to answer any questions.

<comment><USER15> Why isn't the obvious answer to do this change? [https://github.com/pypa/packaging.python.org/issues/564#issu...](https://github.com/pypa/packaging.python.org/issues/564#issuecomment-428553391)

<comment><USER14> Because right now you can _only_ upload individual distributions to PyPI. A release is implicitly created when you upload the first one. If PyPI implemented that change, you could only have one distribution per release.

The proper fix would be to make publishing a release a separate operation. But that breaks all existing tooling and workflows.

<comment><USER16> Make it a configuration option for the package on pypi, then tooling can migrate slowly and at some point it can become the default for new packages. If then at some point someone uses old tooling the downside would be that they might need to do a manual publishing step to actually publish their package, but at least you don't have the problem of publishing before you are ready.

</comment></comment></comment></comment><comment> your-nanny [https://github.com/pypa/pip/issues/5874](https://github.com/pypa/pip/issues/5874)

This seems like reasonable suggestion.

</comment><comment><USER16> shouldn't pip then also look at the hashes (if provided) to determine which package to install? If it worked at one time it should keep working if there are "better" options available now. Consistency is important here.

<comment><USER17> The article links to [https://github.com/pypa/pip/issues/5874](https://github.com/pypa/pip/issues/5874) which proposes exactly this.

<comment><USER14> In fact I'm hoping to get some time this weekend to make a PR implementing this! (I'm assuming the reason you/Donald Stufft haven't done so is time, rather than disagreement over whether this is a good solution or some unseen obstacle to implementing it.)

</comment></comment></comment><comment><USER18> How does R avoid this then?

<comment><USER19> I believe that, for R, releases are synonymous with distributions. The code is compiled on the user's machine (I once had a fortran error message while installing an R package).

If reproducibility matters, you can also use Microsoft R Open to get your packages from a frozen snapshot of CRAN : [https://mran.microsoft.com/](https://mran.microsoft.com/)

</comment></comment><comment><USER20> [https://xkcd.com/1987/](https://xkcd.com/1987/) (edit: this is for anyone who mentions conda in here)

<comment><USER21> How so? Conda is strictly simpler than alternatives, by a solid order of magnitude at least.

</comment></comment><comment><USER21> It's beyond disappointing that the article does not discuss conda and conda's build variant capabilities [0].

Maybe 2 years ago there was still room for honest debate among Python package tooling. There just sincerely isn't anymore.

[0]: [https://conda.io/projects/conda- build/en/latest/resources/va...](https://conda.io/projects/conda- build/en/latest/resources/variants.html#build-variants)

<comment><USER22> As someone who aggressively uses conda, I do think one of its downsides is how heavy it is. I agree that it's a good one-stop-shop if you want to get an environment up and running with no issues. But if you're not doing any scientific computing for a given project, it might help to use something more lightweight.

<comment><USER20> I've had to 'debug' quite a few (ana)conda setups, mostly ones that were used for scientific work, where the whole python setup got so messed up that nothing was quite working anymore, both system and conda pythons. I advise against it.

<comment><USER21> This sounds highly implausible, since conda's number one mode of operation will fully isolate system dependencies into separate environments. It is extremely difficult to misuse conda in a way that would cause this problem, which makes me believe this comment is just trolling.

<comment><USER20> Sorry, no trolling here. The times I've tried to fix the setups I've lost my patience with it and IIRC got things back working by deleting conda and installing packages through manually homebrew (all Macs). But it wasn't on my machine, so I'm not sure how they got into that state. I think it was due to using a wide range of IDE's/environments that all tried to manage their own packages and load paths, combined with perhaps some manual setup.

Probably Conda is fine if you just commit to it, but I think the users who got into these messes weren't knowledgable enough to oversee the consequences of their actions.

So you could argue that the problem isn't with conda, but with the users. However, it seems that for naive users the consequences of using conda aren't clear (enough), which can get them into a mess.

<comment><USER23> Hi! I was the conda dev lead for 2.5 years. We work really hard to make sure conda does the Right Thing in unpredictable, almost hostile, environments. It's a large surface area to guard against, but in general we've been incredibly successful.

I hate to hear that you had a bad experience. If it's at all possible, please provide something on our issue tracker (github.com/conda/conda) we can replicate and write regression tests against. Help us improve on what we've missed.</comment></comment></comment></comment></comment></comment>


Reimplementing a Solaris command in Python gained 17x performance gain from C - <AUTHOR> https://blogs.oracle.com/solaris/reimplementing-a-solaris-in-python-gained-17x-performance-improvement-from-c <comment><USER2> Interesting. Looks like the reimplementation followed a different strategy. Makes me wonder if a C rewrite of the Python script would be even faster.</comment>


Brian Mitchell: JavaScript's Twin from JSConf 2010 - <AUTHOR> http://blip.tv/file/3686402

<comment><USER2> I knew it was going to be Lua. Why not have a Javascript-y front end for Lua? There's also a dual Python-Lua environment. (Closely integrated VMs)

</comment><comment><USER3> It was off the cuff but it was a great conference for open topics. There was a very positive vibe after this which shows how much the JavaScript community has matured (less of an inferiority complex at least). Of course, I had to beat them all with my monopod first.</comment>