Freeing my tablet: Android hacking, software and hardware - <AUTHOR> https://www.thanassis.space/android.html <comment><USER2> Impressive work on the rooting procedure, but...

>For the TL;DR crowd: I wanted to run a Debian chroot in my tablet;

I'm sad that there aren't more people looking to run a mainline kernel + Linux distro on their tablets. It's definitely possible but a lot more difficult.

<comment><USER3> Not quite the same, but this guy describes a way to install Debian as the main OS (but using the kernel from the original firmware) and then running the _Android_ system as a chroot within Debian:

[http://whiteboard.ping.se/Android/Debian](http://whiteboard.ping.se/Android/Debian)

<comment><USER4> Thank you so much! Your link and this article are just what I needed to take back control of my sony compact camera. I will try it soon now.

<comment><USER5> _Please_ blog and document the process. I'd love to read it.

<comment><USER4> With pleasure but only if it's a success.. I want to be very careful so it may takes time.

There is work in progress for sony camera on android (mine is too old): [http://www.personal-view.com/faqs/sony-hack/hack- development](http://www.personal-view.com/faqs/sony-hack/hack-development)

</comment></comment></comment></comment></comment><comment><USER6> I just got a Mi5, and I had to jump through Xiaomi's unlocking hoops. What an ordeal.

You have to register for a Mi account and a Miui forum account and then request unlocking privileges for it. This is apparently a manual process: approval can take anywhere from two days to several weeks. Having more internet points on their forum[1] may or may not speed up the process. Approved Mi accounts can unlock 1 device per 30 days.

Once you've got the unlocking privs, you can download a Windows (and only Windows) executable that performs the unlocking process (it comes bundled with the fastboot/adb executables). It checks that your phone is logged into the same Mi account as one part of a completely opaque validation.

The official documentation, as it were, says you need their "global development" OS running on the phone, but I'm pretty sure it worked with the non-development version, which is good because updating to development failed with a non-descript error.

Once I unlocked the phone -- using a friend's Mi account which had unlocking privs, unbeknownst to him since he never got the notification -- I had to install TWRP (the most popular alternate recovery image). The official versions did not work (either crashing or nonfunctional). After about a dozen recovery images from various file hosters -- exactly where I want to acquire core system software -- I stumbled on one that worked, albeit defaulting to Chinese.

I suppose the fact that I did not brick the device speaks to the robustness of Android's bootloader/recovery/system separation.

[1] A forum which goes to ridiculous lengths to feature these points. Points for replying, points for getting replies, bounties, etc etc. Users posting useful files restrict the link visibility to people who have answered in the thread, which leads to threads with hundreds or thousands of one-word "Thanks" posts, each taking up 200px of vertical space due to the amount of "badges" the forum has. It's insane. Is that a Chinese thing? Or just how the kids do it these days?

</comment><comment><USER7> It has been over a decade since Torvalds rejected GPLv3 licensing for Linux ([https://lkml.org/lkml/2006/1/25/273](https://lkml.org/lkml/2006/1/25/273)) and it turns out that Stallman was right.

<comment><USER8> Right about what? The Linux Kernel has gained tons of work back by vendors using it for products that would be replaced with functionally identical non- GPL3 components.

Torvalds' decision was extremely practical and has lead to huge gains for Linux users even if (ahem) not every company gives back.

<comment><USER9> Hence why Google is happily replacing every GPL component left in Android with MIT BSD ones in each Android release, GCC has already been shown the door.

In Fuchsia, they are even replacing Linux by their own micro-kernel, BSD licensed.

How much contributions has Sony given back to BSD from their PS4 BSD distribution?

<comment><USER8> They've made a handful of contributions back. Arguably, a lot of the changes they've made they couldn't share anyways (AMD licensing agreements) or were built on top of it as applications.

FWIW: I, for sure, appreciate the GPL3 and copy-left licenses.

</comment></comment></comment></comment><comment><USER10> _> Golly gee, Mr Google, that's a lot of partitions_

More like Qualcomm. They like this mess of partitions, not Google.

 _> I am NOT a bad guy!... I just want to remain in full control of my OWN hardware..._

Yeah, they don't want to respect that. And if you care about that, your choices of hardware become pretty limited. Let alone if you want to have open drivers for key components like GPU and the rest. In practice, Google's Nexus devices tend to be one of the best choices (i.e. such as Nexus 5 and Nexus 7). Not sure what the situation with Pixels is.

<comment><USER11> >> _I am NOT a bad guy!... I just want to remain in full control of my OWN hardware..._

>Yeah, they don't want to respect that.

Honest question: if you were responsible for platform security, how would you tell apart the bad guys from the legitimate owner asserting control over their hardware? My imagination might be failing me, but I can't see a scenario where making it easy for the owner to get root doesn't also make it easier for malicious third parties to subversively add rootkits.

The only workable solution I could think of is to sell factory-rooted models for the advanced users and a locked-down version for everyone else, but I suspect the market is not large enough to support an additional SKU.

edit: _fastboot oem unlock_ is also fairly reasonable compromise, when available, but I don't think many users would know what a unlocked padlock icon when booting means in any case, and would happily use the device

<comment><USER12> > _The only workable solution I could think of is to sell factory-rooted > models for the advanced users and a locked-down version for everyone else_

How about Google's hardware "dev switch?" Originally they hid it inside the Kensington lock, but now it's purely a software switch.

[https://www.chromium.org/chromium-os/developer- information-f...](https://www.chromium.org/chromium-os/developer-information- for-chrome-os-devices/samsung-sandy-bridge#TOC-Entering)

When the switch is on it also displays a scary warning for 30 seconds when the computer boots that "this software cannot be trusted."

</comment></comment></comment><comment><USER13> > I started looking at the various offerings, and being a nerd of a frugal > nature, decided to only look at the best HW bang for the buck, completely > ignoring the SW aspects.

I think this is a very important point that people tend to overlook. One that I've definitely stated to appreciate in recent years.

For a similar example, sometimes people talk about the cost of ownership of a PC. Whenever you bring that up, someone will mention how it's really cheap to build your own PC. And although they're technically correct, since it's really cheap to build your own PC, it usually means you're left on your own for maintaining it as well.

I can't shake the feeling that this case is similar. Although I certainly enjoy tinkering with stuff, I think there's a strong argument to be made for things that "just work", or things that can be easily hacked. I have a hard time imagining that the real cost of ownership of this device for the owner ended up being higher than if they had purchased a more open device.

With all of that being said, I found this to be a very interesting read, and I'm thankful that the author took the time to write up their experience.

</comment><comment><USER14> Is there a phone or tablet you can buy which comes fully unlocked and rooted by default?

Something I can install KDE Neon to?

<comment><USER15> Nexus devices are the closest you'll get - can be unlocked by a vendor supported, fully reversible ADB command.

<comment><USER16> From what I've read and seen, nexus devices are in general best supported for development (ie: custom software). I've had some success running custom roms on an old HTC phone, various Samsung devices (the original i9000 Galaxy, a Note2 and 3).

After looking for a while into various phones, I ended up getting a Sony Z3+ -- as it combines many things I want (water proof, memory card slot) with a pretty strong commitment from Sony:

[http://developer.sonymobile.com/open- devices/](http://developer.sonymobile.com/open-devices/)

Ironically, I've yet to actually experiment with different roms and other software on my Sony phone, but there it is.

<comment><USER2> What happens when one day [http://developer.sonymobile.com/unlockbootloader/](http://developer.sonymobile.com/unlockbootloader/) isn't available any more? (see also similar pages for Xiaomi and Motorola)

The code "protecting" the bootloader is good enough that it could be very, very difficult to unlock your bootloader without official support.

<comment><USER16> I'm not particularly happy with Android as a platform in general - it is way too "open on paper, closed in practice" \- and I agree abandonware is a real concern. However, if you just bought a device, it seems a bit less likely the unlock page will disappear before you manage to unlock your device. And once unlocked, it'll stay unlocked?

I feel this is a little bit different from the "zune drm hole" where you "buy" (rent) content, but might loose all access once the provider goes under/closes shop. (See also: the case of Amazon recalling copies of "1984" it had sold as an e-book without a proper license).

<comment><USER2> > However, if you just bought a device, it seems a bit less likely the unlock > page will disappear before you manage to unlock your device. And once > unlocked, it'll stay unlocked?

What if I buy it from eBay 5 years later?

<comment><USER16> I agree that it leads to a new kind of "planned obsolescence", and would prefer something more open.

</comment></comment></comment></comment></comment></comment></comment><comment><USER17> Has anyone tried GNURoot? As I understand it's a Linux chroot without needing your device to be rooted:

[https://play.google.com/store/apps/details?id=com.gnuroot.de...](https://play.google.com/store/apps/details?id=com.gnuroot.debian)

<comment><USER18> I've used it. It's a reasonable solution for a subset of the things you'd want to use a Debian chroot for.

</comment></comment><comment><USER19> What would be the feasibility of making a Snapdragon based phone and running mainline Linux + freedreno on it?

<comment><USER20> Not profitable at the least. You need to think like the average consumer: "Why should I pay $800 for this ugly 'open source' phone (whatever that means) when this nice, attractive looking Samsung phone is only $50 plus $20 a month?"

<comment><USER19> Sorry, I wasn't clear enough. I wasn't asking about selling the thing. Just building it.

</comment></comment></comment><comment><USER21> This is why I still come to HN

<comment><USER22> +1. Fantastic read.

</comment></comment><comment><USER23> Great write up, thanks for that.

I like having small devices, and I like Linux. My solution was to buy a small HP stream 11, and just put Linux on it. Not impressive technically, like the author's work on his tablet, but an easy way to get Linux on a $190 small device. I like to travel with my Linux stream: fully capable for work, using the web, and cheap enough that I don't worry about losing it.

</comment><comment><USER24> Good article. I'm also in the process of getting control of my tablet, by building a custom ROM for it. However, it's not possible to unlock the bootloader as described in the article as Lenovo has decided that it should not be that easy, so I need to take care of that first.

It would have been fantastic if the tablet in the article had a locked bootloader, because I really want to learn how to unlock these bootloaders from scratch. I'm just not comfortable by downloading random binaries from questionable filelockers and loading them on my tablet, I want to know how they are produced so I can do it myself.

I'm reading everything I can find, but the information on this subject is very limited and consists most of "flash these files" and absolutely no informtion how they were created.

Does anyone have good resources on the process of unlocking bootloaders from scratch?

</comment><comment><USER25> Is there some kind of authentication for the serial port in the headphone jack? It sounds like a security risk to me.

<comment><USER26> Generally speaking, if you have physical access to device, all bets are off. Which when you think about it is the point of the whole article.

<comment><USER27> People are a bit more squeamish about applying this logic to mobile devices. After all, iPhones remain attack-resistant with physical access.

"Secure against physical access by all but extremely determined attackers" is worth aiming for. Even if it's just a measure against the resale of stolen devices.

</comment></comment></comment><comment><USER28> Just as an aside: I have a Ubuntu chroot running on the Ubuntu Touch/Phone. It's running mutt, taskwarrior, vim, git, ledger and a few other bits nicely. Still a work in progress, but it's nice to be able to do that instead of make the whole device writable. That's the other option and it breaks OTA updates and potentially more.

</comment><comment><USER29> Some more resources for Debian on mobile devices:

[https://wiki.debian.org/ChrootOnAndroid](https://wiki.debian.org/ChrootOnAndroid) [https://wiki.debian.org/Mobile](https://wiki.debian.org/Mobile)

</comment><comment><USER30> mobile wise when they started doing that i quit there was no point in using android. there is always sec testing but it is not as modular as it was. there is security and then there is economic security.

</comment><comment><USER31> Most of author's difficulties simply stemmed from not understanding how modern android works and boots. Easily fixable by reading more about it.

And no, nobody hates guys like you. Problem is that what you want to do is very similar to what malware might want to, and since malware is more common than guys like you, choices are made that way.

<comment><USER22> Read the Conclusion section of the article. He uses Chromebooks as an example, to argue - quite convincingly - that we could have both security and freedom to tinker.</comment></comment>


Awkward – A Node.js-based terminal emulator - <AUTHOR> https://github.com/iostreamer-X/Awkward <comment><USER2> This is good, but the data structures - eg arrays of arrays - don't really match the underlying data. Imagine what 'ip addr' or 'ifconfig' would look like - they output paragraphs rather than lines, so scraping lines wouldn't produce good output.

It'd be better - and FAR more work - to make an object pipeline based powershell equivalent for Nix, based on JSON. You could write cmdlets in any language that outputs JSON and do better at this project's goal of avoiding scraping.

        ps node | select pid

instead of:

        ps().find(function(line){ return line[3] === 'node' })[0]

Since the fields have keys, you avoid magic numbers and the code's easier to read.

(you could of course alias those to make 'pidof node' like Linux distros do)

The end result would be much better than Powershell, since it would use JSON and not be tied to .net languages.

<comment><AUTHOR> That sounds so good and better. Thank you, I will work on it.

<comment><USER3> Consider:

wrapPs( "ps -ef" ) => [ {pid: 1234, name: "foo.sh" }, ... ]

wrapLs( "ls -al" ) => [ ... ]

(not the syntax but the idea). You want to allow interested individuals to write the wrappers "forcefully" against uncooperative maintainers (separate people, separate ideas, separate motivations, separate release cycle, etc).

Eventually, some enlightened shell-command owners might add "ls -al --json" or "ps -ef --json", "git stat --json" which removes the need for a wrapper script, but the wrapper script allows innovation peripheral to the core without affecting the core until finally due to overwhelming support the core is extended with something "good" and agreed upon via consensus usage.

For the short term, perhaps if "ps" and "ps.wrap" exist in your path, your awkward shell can inject the "* .wrap" automatically around the given command in a way that doesn't require a lot of typing, either automatically, or via shortened syntax.

(ps -ef).map(...)

!(ps -ef).map(...)

{{ ps -ef }}.map(...)

I don't know but eventually you'd want to get to a point where you're bridging between the interactive use case (ps -ef printing a nice text table) and the pipeline use case (ps -ef --json letting you easily access different fields), maybe {{ ps -ef }} either calls * .wrap if it exists or if it doesn't exist calls it with --json (this assumes the command supports --json directly)

Regardless, this idea has some legs and I look forward to seeing how much traction it gets.

</comment></comment></comment><comment><USER4> Uh...

      ps | awk 'NR>1{ print $1, $4}'

I'll give that the syntax of these shell programs take some work to understand but they're hard to beat for the purpose they serve.

If you're parsing plenty of JSON these days I find 'jq' indispensable. It's basically awk/sed for JSON.

I would just be sure to study those programs as they took quite a bit of effort to make them fast. And there are plenty of alternatives with extended syntax and features.

Best of luck.

<comment><USER5> I'm sorry but that seems to be some of the least intuitive code I have ever seen. What are the practical applications of knowing AWK over knowing JS? (Besides the shell voodoo). I'm not trying to start a language war, I just want to understand why learning a completely new set of grammars with a very limited domain would be worth the effort.

<comment><USER6> Awk is actually very intuitive once you learn the basics (should take about an hour). Awk is essentially a line-oriented pattern matching language, that is, a collection of patterns -> actions that are applied to every line of input. Syntax is very similar to C, but feels much more lightweight.

To decompose the example above:

      NR > 1 {
         print $1, $4
      }

There are two parts to this, the pattern (the part outside the brackets), and the action (the part inside the brackets). Every line parsed is split into fields and stored in $1..$NR, where NR is the number of fields. The entire line is also available in the $0 variable. The default separator is the space character, though that can be changed.

So, knowing the above semantics, the meaning of the above example should be clear: If the number of fields for this line is larger than 1, print the first and fourth field of the line. It's a very powerful paradigm, and you can do crazy stuff with Awk. Examples are an x86 assembler [0] and a SASS-style CSS preprocessor [1] (plugging myself there a bit).

Unix coreutils are very powerful once you're familiar with them.

[0]: [http://doc.cat-v.org/henry_spencer/amazing_awk_assembler/](http://doc.cat-v.org/henry_spencer/amazing_awk_assembler/) [1]: [https://github.com/deuill/fawkss](https://github.com/deuill/fawkss)

<comment><USER7> Not intuitive enough apparently ;)

$NR stores the number of records (or the current record number) not the number of fields (that's $NF). In the example the "NR > 1" is meant to exclude the header line from the output.

<comment><USER6> You are entirely correct -- I got tunnel vision/brain fart and mixed NF and NR up, newbie mistake!

NR is initialized to 1 on the first line, and is incremented for every subsequent line read. So, in this case, the above pattern will match any line after the first.

</comment></comment></comment></comment></comment><comment><USER8> Normally I have a severe allergic reaction to anyone trying to do anything serious with Javascript, but this looks like a legitimately good idea and a decent innovation on how shells work.

I'm beginning to think that it isn't Javascript that I hate, it's the way people use it (i.e. to build bloated, slow, terrible webapps). But this is actually pretty cool. JS is shockingly decent at text processing so repurposing it as a shell language kinda makes sense.

<comment><USER9> It's a bit verbose for a shell language (all the async stuff tends to just get in the way in my experience using it like that), but it's great in the sense that everything can be treated as "text" and the functional bits work well when parsing over lines.

</comment></comment><comment><USER10> 49 comments and no one points out that this is in no way a terminal emulator? It's a (rather rudimentary) shell.

</comment><comment><USER11> Sorry I'm late to this, but I'm a big AWK fan, I use it daily and have done so for years. For sysadmin things its great for doing data manipulation in shell scripts.

I get that Awkward is a shell, and I can see the use. But it seems very clunky to use.

This AWK script was posted in this thread

          ps | awk 'NR>1{ print $1, $4}'

It takes the output of the PS command, tosses the header away and then prints the value of the first and fourth fields out. This data can be piped to other programs for additional processing.

The NR>1 idiom is pretty well known after you look at a few example scripts that use it. To replicate the scrip that the Awkward video uses, it's just

          ps | awk '{ print $1, $4}'

It's shorter than the Awkward example. I think it's simple, for every line you get from PS print the first set of text and the fourth set of text. Not sure what Awkward would need to do to skip the header.

@erikrothoff makes a point about whats the value in knowing AWK over JS. I'd like to think that it gives you a powerful portable tool that can be used as either a stand alone application or as part of a shell script. If you are a Windows user, then it's not going to fit as well, but if you are a Unix/Linux/OS X user, it's a worthy tool to add to your arsenal.

The syntax may be "some of the least intuitive code" but investing about 20 minutes can fix that. There are lots of tutorials that can give you a quick leg up. There are also a few "one line AWK scripts" collections that are good samples.

One of the most powerful things about AWK is the ability to use regular expressions to see if you want to process the record. The ability to go through a file, pluck out the record(s) and process them is very nice.

        ps | awk '/astro/ {print $1, $4}'

will give me any user that has astro in it. Not sure how that would work in Awkward. The use of regular expressions like this is what makes AWK a go to language for text processing.

Not here to add to the AWK vs Javascript debate, but wanted to add AWK is a powerful tool, having it in your toolbox is a good thing.

</comment><comment><USER12> Hey,

This is very cool. It does remind me of powershell in it's aims & inspiration. But agree with nailer that using json (or javascript data structures) would hopefully end up with a better, more cohesive shell in awkward. I no longer use Powershell (I got sick of the microsoft ecosystem)

I'm not sure what format Powershell uses to pipe structured data around, but it seemed to be some kind of ad-hoc 'whatever-they-came-up-with-at-the-time' mish mash, instead of using say, JSON, and leveraging the power of arrays, dictionaries et al. In fairness json was just starting when I was using Powershell, so it wouldnt have been on their radar.

One question, How are you turning output from ls into arrays? Or is ls just a javascript function that behaves like ls?

Oh, also, what strategies do you envision for wrapping arbitrary commands, or creating an ecosystem of wrappers? Perhaps some kind of plugin system?

I am currently also getting excited by hyperterm, and wonder if there is some kind of fusion there with your project (that is getting _alot_ of traction).

Cheers for the cool software!

<comment><USER12> So Ive actually been looking through the source for awkward, and it led me down the rabbit hole to shelljs, and cash.

shelljs actually reimpliments alot of shell commands in javascript, eventually mutating the output of the functions into flat strings here: [https://github.com/shelljs/shelljs/blob/master/src/common.js...](https://github.com/shelljs/shelljs/blob/master/src/common.js#L93)

The point being that the outputs of each of these functions is available (technically, it may require refactoring to get at the output pre-stringified) without having to parse formatted text.

Alot of work has been done here.

</comment></comment><comment><USER13> Why do people start new projects in CoffeeScript?!

<comment><USER9> Because it's a language just like any other? It's not like it's deprecated or anything.

<comment><USER13> It's irrelevant given TypeScript and ES2015+.

<comment><USER9> I'd say it's far from irrelevant.

The fact that it uses whitespace instead of braces is a big plus for people used to that, and generally you write less coffeescript compared to TS or ES. Plus it has things like switch expressions, list comprehensions, chained comparisons, the safe nav operator, and a ton of other little features that either don't show up in TypeScript or ES.

And it's very simple (no configuration needed), and compiles to very readable javascript which means converting to plain JS is very easy (hell, there's even a purpose built tool called decaffeinate which will translate it directly to ES2015)

You might not like the language, I don't really like it all that much anymore either, but it's not irrelevant, and for a small personal project like this it seems like a great choice which is easy to switch away from if needed.

<comment><USER13> Well, its declining popularity makes it irrelevant.

</comment></comment></comment></comment></comment><comment><USER14> I wonder how long it will take before something like this gets implemented as a `Hyperterm` [0] plugin.

I think it could be quite useful to combine `npm` packages with shell commands and `map`, `filter`, etc, in your terminal.

The only thing I am unsure about are the underlying representations of the shell output. Arrays of arrays are not very semantic, so the code that parses them would not be very pleasant.

Given a shape like:

      [
        [ 'PID', 'TTY', 'TIME', 'CMD' ],
        [ '13750', 'pts/14', '00:00:00', 'bash' ],
        [ '25193', 'pts/14', '00:00:03', 'node' ],
        [ '25283', 'pts/14', '00:00:00', 'sh' ],
        [ '25284', 'pts/14', '00:00:00', 'ps' ]
      ]

I would probably like to be able to apply some kind of transforms by default so that I can work with:

      [
        { pid: '13750', tty: 'pts/14', time: '00:00:00', cmd: 'bash' },
        { pid: '25193', tty: 'pts/14', time: '00:00:03', cmd: 'node' },
        { pid: '25283', tty: 'pts/14', time: '00:00:00', cmd: 'sh' },
        { pid: '25284', tty: 'pts/14', time: '00:00:00', cmd: 'ps' }
      ]

Though if this was the case you would want some way of applying the right transform depending on the arguments passed into the command.

I guess the bad thing is that there is currently presumably no way of using other shell features like pipes.

Also, if you're going to start outputting different shaped data, then I think you should consider hooking this all up to `flowtype` [1] and creating type signatures to provide DX [2]

[0] [https://github.com/zeit/hyperterm](https://github.com/zeit/hyperterm)

[1] [https://flowtype.org/](https://flowtype.org/)

[2] [https://news.ycombinator.com/item?id=12129026](https://news.ycombinator.com/item?id=12129026)

</comment><comment><USER15> Like. But PowerShell still rules. Wish it was oss and x-plat

<comment><USER13> Microsoft seems to be working on porting PowerShell to Linux already [0] and there's a partial port using Mono [1]!

[0]: [https://windowsserver.uservoice.com/forums/301869-powershell...](https://windowsserver.uservoice.com/forums/301869-powershell/suggestions/11689284-powershell- core-for-linux)

[1]: [https://pash-project.github.io/](https://pash-project.github.io/)

<comment><USER16> Oh, I see that pash was revived. It had been abandoned since 2009 (I think).

</comment></comment></comment><comment><USER17> FWIW, I've been using Groovy for this purpose to encapsulate processes in *Nix and Windows. Not perfect, not lightweight, but happens to get the job done.

<comment><USER18> Apache Groovy's perfect for these kind of tasks where it doesn't need to be lightweight. Just don't use it for building actual applications -- build them in a decent typed language like Java, Scala, or Kotlin, then use Groovy as a multi-OS version of Bash to automate builds, tests, and runs.

</comment></comment><comment><USER19> Neat hack! I like the idea. Initially when I read the title I thought it would be yet another 'make my shell behave more like my preferred language' type of project. Those while fun to code and play with aren't really useful in the long run. This though might just be useful.

<comment><AUTHOR> Thank you!

</comment></comment><comment><USER20> I'm imagining a world where coreutils and built-ins all respect an environment variable, say _STDOUT_JSON=1_ , and would output JSON. That would be a great start. Especially if the JSON-Schema could be output with a new built-in, _manj_.

How Herculean of a task is this?

<comment><USER21> Not a good idea. Imagine that you invoke some program A that in turn invokes a coreutils program B. Imagine that A has been written before today (in other words, it is not aware of the STDOUT_JSON technique). Since A and B inherit the STDOUT_JSON environment variable, A will receive JSON. But it is not expecting this, so it will probably crash.

<comment><USER20> Thanks for that answer, that lead me to learn and understand more about standard streams and file descriptors than I ever cared to before! That's a good thing. :)

So then, what about this approach: since the de-facto standard streams are STDIN, STDOUT and STDERR using file descriptors 0, 1 and 2, respectively, are there three more reserved numbers (I couldn't find any in my google-glance) that could be used for, say, JSONIN, JSONOUT and JSONERR?

I think the heart of the matter is: unless it becomes part of the POSIX standard, is there any hope on agreeing on an informal "UNIX philosophy" sort of approach that would allow coreutils and shell built-ins to accept and pass structured responses in any kind of object form?

</comment></comment></comment><comment><USER13> I tend to like ShellJS a lot recently. And yesterday I found this: [https://news.ycombinator.com/item?id=12125660](https://news.ycombinator.com/item?id=12125660)

</comment><comment><USER22> Making all in a terminal an object is very interesting!! I'm curious about applicability though. What can you do with this that you wouldn't normally do in a traditional term?

</comment><comment><USER9> how can you apply flags to the command?

For example, I use `ps -ef` a lot, how would I do that with this system?

<comment><AUTHOR> This seems to work, (ps -ef)().map((a)=>{return [a[0],a[7]]}).forEach((a)=>{console.log(a)})

<comment><USER9> Ah, put the whole command in parens!

It might make sense to put a note in the README about that, unless it's already there and I missed it somehow...

<comment><AUTHOR> I am sorry, it slipped from my mind. I will add it now.

<comment><USER23> Just a note, _iostreamer_, you can use `command.parse` in Vorpal to pre-append parens to every command before it's evaluated (your project is written in Vorpal).

BTW, love this idea.

\- the Vorpal guy

</comment></comment></comment></comment></comment><comment><USER24> The following comment could apply to hundreds of similar submissions to HN in recent years.

Many people -- many of them young people I suspect -- have invested a lot of time learning Javascript. It has become a very popular language.

But it remains to be seen whether Javascript will be as long lasting as the UNIX shell and standard utilties. Historically speaking, computer languages have been known to fall into and out of popular usage.

For people who invested a lot of time learning the shell and ubiquitous utilties such as AWK, it appears the investment has paid off. I'm not too worried about the terminal disappearing any time soon.

How long until the next submission that aims to abtract away the need to learn how to use a UNIX terminal -- directly.

There are probably hundreds more on Github alone. What if we conslidated them all in one place: 1001 attempts to abstract away the need to learn UNIX.

By no means am I suggesting these attempts have not been successful.

What I'm suggesting is that the need for them will not abate. It could be that UNIX terminals, and programs like AWK, are not just a fad.

<comment><USER14> Perhaps there are features from other programming languages that should exist in shell programming?

For example, I've always felt that the fact that almost everything in shell programming is a gigantic string is really bad. It would be much nicer if all commands had type signatures, and therefore piping commands into each other could throw type errors if it does not make sense.

This would also allow much better feedback when stringing together commands, since the shell IDE would be able to recommend commands that act upon a particular type. For example, if a command produces output that could be considered to be N > 1 lines long then `wc -l` would be available, however if this isn't the case it does not make much sense for it to be recommend to the user.

<comment><USER25> Like in PowerShell?

<comment><USER14> I've not used it before and I presume it is Microsoft Windows only.

</comment></comment></comment></comment><comment><USER21> This is great! I'll just let this evolve a bit, and wait for broader community support, and then I'll happily replace bash by this shell!

<comment><AUTHOR> I would love if that happened :D</comment></comment>


Ask HN: How to maintain focus during long, critical-thinking intensive, tasks - <AUTHOR> Hi HN,
As my career has progressed and I have been given larger and more complex software engineering assignments, I am finding that my ability to focus seems to diminish in proportion to the complexity/longevity of the task at hand. How do you all achieve prolonged periods of focus throughout the day? I am especially curious what those who are in a startup, in-office, environment would say as these tend to be very "personal" (i.e. I can't lock myself in an office).
Thank you! <comment><USER2> You should start with reading _Deep Work_ , by Cal Newport. And recognize that there is a limit to just how long you can concentrate deeply. Be conscious of diminishing returns.

[http://calnewport.com/books/deep-work/](http://calnewport.com/books/deep- work/)

<comment><USER3> I really loved the book in combination with the book Flow by Mihaly Csikszentmihalyi. In Deep Work are several strategies explained to get into a state of flow by doing deep work. [0]

\- [0] [https://www.robinwieruch.de/lessons-learned-deep-work- flow/](https://www.robinwieruch.de/lessons-learned-deep-work-flow/)

<comment><USER4> In addition to the above, a more general recommendation would be to look into meditation.

Sam Harris's _Waking Up_ , along with an app like Headspace, would be great places to start.

</comment></comment></comment><comment><USER5> I chunk all work down to 2 hours max. Keep chopping it up until I can get it small enough.

Then I completely focus for the period of time needed, no other distractions, 15 to 120 minutes. It's similar to the Pomodoro method, except with a max of 2 hours. If I feel the urge to do something else, I jot it down to be done in the break.

I then force myself to take a mandatory break after the estimated time is over.

If I expect a distraction during the needed time, like a meeting or lunch, I do something else that fits the time slot.

If I'm uncertain how long a task takes, I put aside 1-2 hours to hack a disposable prototype or do research.

</comment><comment><USER6> There's always limit on how much one can focus. Here's what I do to compensate:

1\. Write everything down, in some sort of structured form.

2\. Take breaks.

3\. Return to writeup, think about it some more.

This scales from smaller two day things to multiple month projects; see Rich Hickey's talk, which focuses more on the latter: [https://www.youtube.com/watch?v=f84n5oFoZBc](https://www.youtube.com/watch?v=f84n5oFoZBc)

</comment><comment><USER7> Early in my career, I had a similar frustration. I took a paper notepad and documented everything I did. I planned on doing it for a couple of weeks. On the afternoon of the second day, I sat down with my manager and showed that the longest period I had without structural interruption was about twenty minutes. By structural interruption, I mean things that require response and cannot be tuned out (and not things like loud coworkers). My responsibilities did not change, instead other people's authority to interrupt me at will was reduced.

That was many years ago. These days, I pull all-nighters in part due to habit from working in environments where all-nighters until it is done (for some definition of 'it' and 'done') were normalized, but in part because I know that my focus before about 16:00 is often limited and increases throughout the evening and into the morning. I'm not recommending all-nighters, because the fact it works for me has no bearing on it working for you.

I am recommending figuring out what works for you based on what works for you and not based on what works for someone else.

Good luck.

</comment><comment><USER8> Do work that you enjoy and you should have no problems. When you run out of focus after 10-12 hours, the only cure is to go to bed.

</comment><comment><USER9> take short breaks ...

for me 5 minutes after every 40 works well ...</comment>


Ask HN: Anyone Hiring Remotely? - <AUTHOR>

HNers,
Excuse the fake account but I didn't want to use my normal one for the sake of privacy.
I'm currently looking for programming or development work. I'm highly productive in Django, Python and jQuery.
I've also worked extensively on App Engine and know its pros and cons (and possible workarounds) well if you happen to be on that platform.
I'm more comfortable with back-end work but I've recently shipped several complete development packages for teams that I've been working on.
I'm located outside of the US right now and am 12 hours off US time give or take. If you're looking for an extra set of hands and someone who works autonomously and quickly I'd probably be a good fit for your team.
I can be reached at remotely.hn [at] gmail if you're interested in discussing further.
Thanks! <comment><USER2> I've worked remotely almost exclusively for 5 years now. For a handful of projects I've gone onsite for a day or two a week but those are the exception.

The key is to build your reputation and network like crazy. Get to know everyone you can. Work to become _the_ guy for technology X. And get out there and pitch on remote projects and ideas that you do come across. Once you knock those out of the park, word gets around, so people will start coming to you. Even you can't do the project (skills, time, etc), try to make a solid recommendation for them. You'll still score some points.

It took ~2 years for that to happen to me, but I rarely go out hunting for work now. I make my presence known regularly and the rest pretty well takes care of itself.

</comment><comment><USER3> If you're soliciting work, why on earth wouldn't you use your normal HN user name? I don't get it.

<comment><USER4> Perhaps they don't want their current employer to know they're on the market.

</comment></comment><comment><USER5> Kind of off topic, but I'm curious how difficult it is to get remote (non- contract) development Jobs these days? I'm not currently on the Job market, but in a couple of years, I plan to move back to the place I was born, and there are not many, if any, great companies to work with there for a software developer.

I guess what it boils down to me wondering if it is easy or hard to find remote development positions and going forward is it becoming easier to find remote positions as a software developer.

<comment><USER6> I'm going to be working remotely for the next year. I currently go into the office in the morning and work from home in the afternoon. A couple weeks ago I asked my manager if I could work from New York for a year and he said sure. I've been at the same place for almost five years now, in large part because of stuff like this. They know me and trust I'll do my job. It would probably be much more difficult to get a setup like this right off the bat.

</comment></comment><comment><USER7> Mozilla's web development team uses Python and Django, and has several remote workers. I work remotely for Mozilla; my own team has workers in Mountain View, Toronto, Seattle, Vancouver, Boston, Pittsburgh, and Paris.

See <http://www.mozilla.com/en-US/careers/> and you can email me (see profile) if you have any questions or want an introduction to someone.

<comment><USER8> If that doesn't work out, I know Canonical/Ubuntu used to have a similar deal, and it looks like they still do:

<http://webapps.ubuntu.com/employment/canonical_ISDFWD/>

</comment></comment><comment><USER9> It would be great if this turned into a "Who's Hiring Remotely" thread.

</comment><comment><USER10> I'm head of integrations at FreshBooks and I am looking for agencies to outsource projects to (PHP, Rails, and Python). Unfortunately, I am not looking for sole freelancers as I want the agency to manage the product management, design, implementation, testing, and validation. However, if you have a group of people you frequently work with who can cover all of that, then drop me a line.

sunir <-> freshbooks <-> com

</comment><comment><USER11> I'm looking for a jQuery Expert / User Interface Developer to help me with a small project (paid), working remotely. Email dk401 [at] cam.ac.uk

<comment><USER12> Is that paid work? <http://tomcarnell.com/>

<comment><USER13> used to work for free?

</comment></comment></comment><comment><USER14> I am working as remote consultant/developer for like 4 yrs now. Once you get a good reputation in any sphere looking for new projects isnt a prob anymore. I specialize in Rails/ajax/push/e-commerce and don't really advertise my services for last like year, while have stable deal flow and even employ team of devs that help me. However I work onsites from time to time when needed.

</comment><comment><USER15> For me personally, there was a stretch from 2004 to 2008 when it was really easy to get remote work. This allowed me to travel and work at the same time, which I loved. Businesses were desperate for people with good web development skills, at that time.

Then the recession hit, and everything changed, at least for me. The remote work vanished. I can still find good paying work in New York City, but its all the kind of work where you have to go into the office everyday.

My guess is that the kind of work that can be done remotely has moved to countries that have very low wages. The kind of work that remains is the kind of work that can not be done remotely (or so the client feels, due to considerations of security or firewalls or the need for cross-functional team integration, or for the need of internal training of other workers who are suppose to learn skills from you).

</comment><comment><USER16> I am looking hire front-end designers. Not sure if HN is the right place, but if you know anyone, please feel free to have them contact me at <my HN username> at gmail.com

<comment><USER17> Ditto

<my HN username> at gmail.com

</comment></comment><comment><USER18> Well, if anybody is hiring remotely: I'm a sysadmin-turned-coder with around ten years of Ops experience, most of which I've spent learning how to automate things. So I'm fluent in Ruby (and Rails), JavaScript (and jQuery), as well as anything that might need to be done on a Linux box.

If it sounds like I might be able to help you out, shoot me an email: don at madwombat dot com

</comment><comment><USER19> I work at On-Site.com and my understanding is that we're willing to hire remotely for the right candidate. The OP is probably not what we're looking for, though -- we're a JRuby on Rails and JSP shop (plus jQuery, which he _does_ know), and we'd need somebody who knew both if they weren't going to spend time physically present (in Mountain View, CA) to get up to speed.

</comment><comment><USER20> We're looking for short-term remote Rails help implementing faceted search using solr/lucene or websolr.

</comment><comment><USER21> I'm looking for sproutcore work or any front end dev

Check out <http://quicksnippets.42horizons.com> for a small example and contact details

</comment><comment><USER22> If you need design/front-end coding hit me up at davide {at} 39inc.com

(as a company we actually do more than just design, but the design part is what i'm personally taking care of)

</comment><comment><USER23> 12 hours away - are you in the Northern or Southern hemisphere? (Just out of curiosity). There doesn't seem to be the same remote outsourcing trend here in the Southern hemisphere.

<comment><USER24> Here in South America we are mostly on the same time zone with the US (+/- 3hours) so there is quite a bit of remote work going on.. Since I got here I've met people doing HTML,php,python,sql and some Java for companies all over the US.. I also know of a few large US companies (fortune 100) who are opening big offices over here. Also, countries like Chile & Argentina are given big incentives for startups to move over there.

</comment></comment><comment><USER25> I am looking for rails and JQuery people for remote gigs. You won't be getting rich off of it but get in touch if interested - firstname.lastname@example.com

</comment><comment><USER26> Shameless plug: if anyone needs a quality assurance engineer - feel free to contact me, I'm currently looking for a remote QA job.

<comment><USER27> contact me username @ gmail

</comment></comment><comment><USER28> sent email

</comment><comment><USER29> I am looking for remote android gigs..

My linkedin and visualcv with demos can be found at links at bottom of my blog: http//mobilebytes.wordpress.com</comment>


Joi Ito (MIT Media Lab) interview: Doing nothing is cheap. Failure is cheaper. - <AUTHOR> http://blogs.wgbh.org/innovation-hub/2014/5/9/recipe-success-fail-lot/

<comment><AUTHOR> Text summary here: [http://www.xconomy.com/boston/2014/05/09/innovation-hub- joi-...](http://www.xconomy.com/boston/2014/05/09/innovation-hub-joi-ito-on- failing-to-the-top/)</comment>


FreeRDP 2.0 – A Remote Desktop Protocol implementation - <AUTHOR> https://www.freerdp.com/ <comment><USER3> I'm so glad that the FreeRDP client on Linux works really well. It's one of the pieces of software that made transitioning away from Windows much more convenient as I continue to work with Windows systems and sometimes, tools.

The RDP server is one thing I miss from Windows though. On Linux I haven't found anything nearly as good. It's really surprising to me too - given that a good visual remoting tool allows for such ease in cross platform living.

I suppose there are a lot of reasons for this, but primarily I guess that most Unix natives would consider SSH to be good enough.

<comment><USER4> Not sure if you're aware, so I apologise if you are:

1) X11 is already a network aware display system, so it wouldn't take too much work to run remote.

2) SSH has built-in X11 forwarding with `ssh -X <host>`, when you run commands they will be rendered locally. On OSX this needs an X11 server (XQuartz) to work properly.

3) There are "better" remote desktop solutions like "parsec"[0] which run on Windows and Linux. They offer much lower latency and higher bit-depth. They're designed for games. (obviously this comes at the price of bandwidth).

[0]: [https://parsecgaming.com/features/](https://parsecgaming.com/features/)

<comment><USER5> X11 is network aware, in that is theoretically possible to speak the protocol over a network. However, anyone considering it should be aware that it is (1) extremely chatty, (2) extremely synchronous, and (3) extremely latency sensitive. It only works well on fast local networks. SSH certainly doesn't make it any faster.

In contrast, RDP and VNC type protocols are screen-replicating designs more suited to higher latency connections.

Finally, if you're using RDP, I'd suggest Remmina as an client. It's fantastic.

<comment><USER6> To pile on against X11 over ssh, forwarding on a modern Linux workstation is spectacularly painful. I recently needed to forward a lightweight gtk app from one workstation to another over 10gbe and a single switch hop.

With ssh and X forwarding, my local gnome constantly threw up modal dialogs that the forwarded window was unresponsive, forcing me to keep clicking "Wait" to dismiss the modal, then quickly interact with the forwarded window before the next "unresponsive" modal popped up.

<comment><USER7> Forwarding Firefox over X11 _sometimes_ works. Sometimes Firefox on the remote machine will try to join the session of your local firefox -- but fail and then complain that there's Firefox already running. Adding `--new-instance` helps if Firefox is running in a container and X11 socket file is forwarded to the container but it doesn't seem to help if X11 is forwarded over SSH.

<comment><USER8> Try '-no-remote'. Or was that '\--no-remote'? Anyway, that's what I've used for years and years when needing to run firefox over ssh X forwarding.

<comment><USER7> Firefox is running on the remote, connected to a different X11 session. It is not running on the local. When I use `firefox --new-session --no-remote` on the remote, I see a dialog pop up on my local machine stating _Firefox is already running, but is not responding. To open a new window, you must first close the existing Firefox process, or restart your system._

<comment><USER9> Yes, this is because firefox doesn't allow multiple ff instances to use the same profile at the same time. You can ask firefox to create a new profile (`firefox -ProfileManager`) and then use a seperate profile for the remote instance

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> I'd encourage freerd.com to be more descriptive on the landing page - <USER2> blurb says "RDP Implementation", but it's not clear if it's client, server, or both. As a client, this is only useful to people accessing Windows.

<comment><USER11> It also doesn't mention the supported OSs prominently either...

</comment></comment><comment><USER12> All the Linux remote desktop server solutions are bad enough over a WAN that I typically VPN into my office, then RDC from my home Windows box into my office Windows box, then ssh -X from there into my office Linux box. It's kinda crazy, but it works better than anything else I've found, and I've tried almost all the options mentioned here. I wish there was a better way, but I don't think there is (for me)...

<comment><USER13> Really? I agree "a new remote desktop" makes me suspicious about usage over WAN because most all suck. I didn't have great luck with Windows Remote Desktop, either. We went with TurboVNC/VirtualGL on the server and recommended, I think, TurboVNC on the client (via ssh). I used macOS Screen Sharing.app because it wasn't Java and was more comfortable to use IMHO (with things like screen scaling and fullscreen). Most other VNC clients performed significantly worse. I usually ran the desktop around 1440x900.

This seemed to be the best setup with high latency and low bandwidth situations. We had people spanning the furthest time zone.

<comment><USER14> I've never got this working, but it seems like the best combo, is there an easy GUI way to connect to VirtualGL / TurboVNC and tutorial to do the server side, it didn't look straightforward when I looked a few years ago.

<comment><USER13> Sorry. Our sysadmin had set it up. We relied on OpenGL applications and had multiple users, so we weren't using :0. I _thought_ I got it working (sans OpenGL/VirtualGL) by just installing the rpm, then each user would run `vncserver -geometry 1440x900 :89` (their per-user port) to launch the server, then connect via the client.

We did have issues periodically. Mostly because the workstations were mostly not interactive, but if someone would plugin or unplug a monitor X would change configuration on the next reboot. Or when we had multiple GPUs we'd tweak things. I wish I kept better notes.

</comment></comment></comment></comment><comment><USER15> Microsoft should really consider releasing their official implementation as open source and integrating it with Linux desktop in a way that doesn't suck. Vnc sucks pretty bad. NoMachine sucks a bit less, but doesn't seem to be maintained. RDP doesn't suck at all. I've worked with Windows machines halfway across the globe using it, and while the experience wasn't super snappy, it was usable. I also spent thousands of hours working from home using RDP when I was at MS in 00's. With sufficient network bandwidth, in full screen, the experience is indistinguishable from a local desktop.

<comment><USER16> Microsoft has released[1] the RDP documentation, which is what FreeRDP is based on[2]. Open-sourcing their implementation would not help, since it's deeply integrated into the OS.

I was hoping Wayland would take remote desktop seriously when I heard about the effort, but sadly that was not to be.

[1]: [http://www.microsoft.com/openspecifications/](http://www.microsoft.com/openspecifications/)

[2]: [https://github.com/FreeRDP/FreeRDP/wiki/Reference- Documentat...](https://github.com/FreeRDP/FreeRDP/wiki/Reference- Documentation)

<comment><USER17> How would you propose Wayland take remote desktop seriously?

<comment><USER16> By taking it into consideration from day 1, as a decent remote desktop experience requires tight integration.

<comment><USER15> Wayland doesn't work properly even without these considerations though. Horrible tearing in Chrome when watching video, at least on my machine, crazy high CPU usage too. I had to switch back to X.

</comment></comment></comment></comment></comment><comment><USER18> Has anyone been able to get FreeRDP's Thincast Workstation (based on VirtualBox) product to work?

[http://www.freerdp.com/2019/02/20/hi-thincast- workstation](http://www.freerdp.com/2019/02/20/hi-thincast-workstation)

I know it's still in beta, but it was originally released two years ago and I'm still not able to boot ubuntu's mini.iso installation media in this graphics-optimized version of VirtualBox.

If anyone has been able to test Thincast workstation performance, can they compare it to a Hyper-V linux virtual machine connected to X410 through VSOCK:

[https://x410.dev/cookbook/hyperv/using-x410-with-hyper-v- lin...](https://x410.dev/cookbook/hyperv/using-x410-with-hyper-v-linux- virtual-machines-via-vsock/)

</comment><comment><USER19> I was using (well trying to use) FreeRDP/XRDP for a while to work with my multiple Linux machines from the Windows client. Soon I've discovered that it was not XRDP working for me but the other way around. I replaced it with NoMachine and and never looked back since.

Now I am somewhat curious but frankly from a practical standpoint I have very little motivation to stop doing what I am doing just to check this new one. It will disrupt my routine for the sake of completely unknown outcome which in the best case will let me have what I already have now. And in the worst case - the opposite of sky is the limit.

This is not to discourage developers but just to show what missed opportunity might cost should there be more people like myself.

<comment><USER14> Same, Nomachine has a GUI, and importantly discovery of clients on the network.

The performance is a lot better than VNC.

In theory I could setup VirtualGL and TigerVNC, but it seems like a lot of work, and time I just don't have.

Other clients could do well to see how easy NoMachine makes things.

</comment></comment><comment><USER20> Install process is not straight forward in Ubuntu and it is installed in a not usual binary search path (look at /opt). No complain against the software (it's excellent) but I think this is a small bug for usability.

</comment><comment><USER21> We changed the URL from [https://www.freerdp.com/](https://www.freerdp.com/) to the home page since this project doesn't seem to have been discussed on HN before.

</comment><comment><USER22> I have used NoMachine. Liked the experience while connecting to Ubuntu from Windows client. With client running on Mac & server on Linux - it is Okay experience - not great. Is FreeRDP 2.0 better? Will try it anyways.

</comment><comment><USER23> Unlike others here, I am non plussed by an open source implementation of an old format mostly used by non free operating systems. It doesn't add much.

I would be far more interested by a GPU accelerated server, with a focus on low latency, like parsec: something more or less equivalent, with native support for wayland or xorg would be much more interesting.

This would superset the uses of RDP: you can use parsec for remote desktops, just like you can with RDP except it's smoother and more responsive.

But you can also use games with parsec, something you can't do with RDP, and everything else that requires low latency.

<comment><USER24> Many people, myself included, use this software to connect to remote windows machines that we have no control over. It's incredibly valuable, you're just not the target audience apparently.

</comment></comment><comment><USER25> What's the performance vs Remina?

<comment><USER26> Remmina uses freerdp as its RDP library.

</comment></comment><comment><USER27> I was severely disappointed when it turned out the Hyper-V Linux enhanced session relied on RDP and the experience was pretty bad. Was hoping for something from Microsoft that was a step up from VirtualBox and VMWare in every way and particularly the latency department. Maybe this new release of free is a major improvement?

<comment><USER18> Have you tried FreeRDP's Thincast Workstation (based on VirtualBox)?

[http://www.freerdp.com/2019/02/20/hi-thincast- workstation](http://www.freerdp.com/2019/02/20/hi-thincast-workstation)

I know it's still in beta, but it was originally released two years ago and I'm still not able to boot ubuntu's mini.iso installation media in this graphics-optimized version of VirtualBox.

Interested also if anyone who has been able to get Thincast workstation to boot linux can compare it to a Hyper-V linux virtual machine connected to X410 through VSOCK:

[https://x410.dev/cookbook/hyperv/using-x410-with-hyper-v- lin...](https://x410.dev/cookbook/hyperv/using-x410-with-hyper-v-lin..).

VSOCK could improve RDP latency since it should be faster than a localhost network socket.

</comment></comment><comment> yjftsjthsd-h Changelog: [https://raw.githubusercontent.com/FreeRDP/FreeRDP/2.0.0/Chan...](https://raw.githubusercontent.com/FreeRDP/FreeRDP/2.0.0/ChangeLog)

</comment><comment><USER28> Is there a nice way to get around NAT?

<comment><USER29> A VPN? I would not expose any kind of remote desktop protocol to the internet.</comment></comment>


Princeton IoT Inspector: discovers IoT devices and analyzes network traffic - <AUTHOR> https://iot-inspector.princeton.edu/ <comment><USER2> Disclaimer: I work in an IoT-focused consultancy and work with Fortune 500s in their IoT strategy.

This is a _great_ idea. Not sure the implementation is perfect, but giving consumers more control/visibility over their devices is going to be increasingly important.

I hope we get more companies into this space, but I don't think it's well understood enough to be a viable business. Until then, open source tools like this are going to be great.

If the authors are watching this thread, reach out, I'd love to connect you with some of my customers.

<comment><USER3> Hi! I work for a security startup that focuses on IoT security, and is solving these issues for enterprise. Would love to connect.

</comment></comment><comment><USER4> From the faq:

Security: All data collected from your IoT devices is stored on a secure server at the Department of Computer Science in Princeton University. IoT Inspector transmits data to our server over a secure channel, i.e., HTTPS.

<comment><USER5> If you want to compile it from source I can't imagine disabling this would be all that difficult: [https://github.com/noise-lab/iot-inspector- client/blob/c5b0f...](https://github.com/noise-lab/iot-inspector- client/blob/c5b0f2f3fc3c34ec1a8d7518c75cd9d599f4cbeb/v2-src/inspector.py#L59)

<comment><USER6> or just the little snitch firewall on macos to disable it. that's what i did. sorry princeton

</comment></comment></comment><comment><USER7> The paper preprint behind this tool is here: [https://www.cs.princeton.edu/~yuxingh/static/pets-2019.pdf](https://www.cs.princeton.edu/~yuxingh/static/pets-2019.pdf)

</comment><comment><USER8> If anyone from the inspector team is reading: The audio in the "Installation" video really should be removed. :-D

<comment><USER9> I don't know, I thought the random snot-sucking and throat-clearing gave it a nice, relatable quality.

</comment></comment><comment><USER10> Why is their python app mac only? I guess it probably works on linux or mindows too. Wish they tell you what the main starting class was.

<comment><USER11> Because they're interacting with network stacks using OS-specific interfaces.

      def enable_ip_forwarding():

          os_platform = utils.get_os()

          if os_platform == 'mac':
              cmd = ['/usr/sbin/sysctl', '-w', 'net.inet.ip.forwarding=1']
          elif os_platform == 'linux':
              cmd = ['sysctl', '-w', 'net.ipv4.ip_forward=1']

          assert subprocess.call(cmd) == 0

<comment><USER12> Ah, that good old "all of US use macs, so we have no reason to make it work cross platform". "Aren't you making this for people to find IoT devices?" \- "Yes?" \- "Isn't probably more than half the student body _not_ on apple computers?"

<comment><USER11> In my experience, the % of students at Princeton using Apple computers is definitely north of 50%. However, given that they're students, I doubt they're the target audience for this utility, since they're likely not homeowners (if they're living in dorms, they likely don't have that many IoT devices to be worried about).

The target audience is likely "privacy-minded technically capable home network owners", a significant proportion of whom likely skew towards Apple products.

<comment><USER6> >The target audience is likely "privacy-minded technically capable home network owners"

why would you use apple if you're technically capable? locked down linux would be waaaay more secure(and give you waaaaay cheaper options)

<comment><USER13> > privacy minded technically capable network home owners

Uploads all data to Princeton's servers.

</comment></comment></comment></comment></comment></comment><comment><USER14> Great idea, however would be great to see an option which used the dataset but did not upload your data.

I've read the "FAQ Why must IoT Inspector upload the data to Princeton?" and do understand the reasons for data collection, maybe once it reaches a data saturation point this new option could be introduced (I believe there is a "Start/Pause Inspection" button but that seems to only temporarily pause collection).

Naturally, there's going to be some reluctance in using and deploying a tool that uploads data to an outside party.

</comment><comment><USER15> From the website: "[2018-04-13 13:32 ET] We're still under heavy load. To reduce the likelihood of the Gateway Timeout error, we've reduced the refresh rate of the data -- i.e., information about your devices will be updated roughly once every minute (instead of once every 15 seconds previously)."

</comment><comment><USER16> Including a nice and extensive FAQ on how they collect data: [https://iot- inspector.princeton.edu/blog/post/faq/#data-priv...](https://iot- inspector.princeton.edu/blog/post/faq/#data-privacy-and-security)

</comment><comment><USER17> Uses ARP Spoofing to capture data [https://en.m.wikipedia.org/wiki/ARP_spoofing](https://en.m.wikipedia.org/wiki/ARP_spoofing)

</comment><comment><USER12> Whoever made this site needs to spend just those few minutes more to ensure that images, one of the most basic HTML elements, actually work with a script blocker (wisely) turned on.</comment>


Mozilla is buying Pocket and its more than 10M users - <AUTHOR> http://www.recode.net/2017/2/27/14752686/pocket-read-it-later-mozilla <comment><USER2> Please don't screw this up. Pocket is amazing.</comment>