SQLite as an Application File Format (2014) - <AUTHOR> https://www.sqlite.org/appfileformat.html <comment><USER2> I wish more people would publish data as SQLite databases (if the size permits, of course; usually it does). It's so much more reliable than CSVs, which have at least a few dimensions of significant differences (quoted/unquoted, comma vs semicolon vs tab vs space, headers/no headers, comments). Not to mention that initial data exploration can be done right in an SQLite explorer/browser tool.

<comment><USER3> As much as I love SQLite, and while it is open source, it is a single implementation that AFAIK has no published open specification. The only way to read an SQLite file is using SQLite, and in that respect, it is for many users just as closed as wrapping something in a word document.

CSV isn't perfect, but it provides a ton of flexibility, for example, CSVs can be streamed or support parallel segmented download across a network with useful work possible during the transfer. The format is so simple that it can approach almost free to parse (see e.g. my own [https://github.com/dw/csvmonkey](https://github.com/dw/csvmonkey) ).

CSV is also distinguished in that regular home users with spreadsheet programs can usually do most things a developer can do with the same file. For me user empowerment trumps all other goals in software, including warts. Things like JSON, XML or SQLite definitely don't fit in that category, although I guess SQLite is at least better due to the wide availability of decent GUIs for it.

Finally as a data transfer format, SQLite has the potential to be massively inefficient. Done incorrectly it can ship useless indexes that can inflate size >100%, and even in the absence of those, depending on how amenable the data is to being stored in a btree and the access patterns used to insert it, can leave tons of wasted space inside the file, or AFAIK even chunks of previously deleted data.

<comment><USER4> > _" As much as I love SQLite, and while it is open source, it is a single > implementation that AFAIK has no published open specification. The only way > to read an SQLite file is using SQLite, and in that respect, it is for many > users just as closed as wrapping something in a word document."_

That's an extreme position to take, particularly since the SQLite code is public domain. Furthermore it's one of the formats recommended by the Library of Congress for archival/data preservation:

[https://www.loc.gov/preservation/resources/rfs/data.html](https://www.loc.gov/preservation/resources/rfs/data.html)

[https://www.sqlite.org/locrsf.html](https://www.sqlite.org/locrsf.html)

<comment><USER3> > The only way to read an SQLite file is using SQLite

This part unfortunately isn't a position, it's absolute. It's hard to imagine a situation where as a developer we would not have access to a C runtime or for any reason whatsoever would not be able to use SQLite, but the hard dependency on its code is real, and represents a real hazard in the wrong environment. A super easy example would be parsing data on say, a tiny microcontroller on an IOT device. This can start to hurt quickly:

> Compiling with GCC and -Os results in a binary that is slightly less than > 500KB in size

Open formats at least give you the option of implementing whatever minimal hack is necessary to finish your job without say, introducing some intermediary to do an upfront conversion, and at least for this reason SQLite cannot really be considered a perfectly universal format

<comment><USER4> By 'position' I was _very obviously_ referring to

> _" as closed as wrapping something in a word document."_

Sure CVS makes it trivial to waste your time reinventing the wheel, making your own parser. The situations were there are technical limitations that prevent the use of sqlite are becoming vanishingly rare. (Not to mention the resources necessary to use sqlite is unrelated to how many implementations there are or whether it's 'open' or 'closed'.)

<comment><USER5> 1) For practical purposes the vast majority of platforms that are inappropriate for SQLite are not appropriate for CSV.

2) SQLite has no standard. The same is true for CSV in practice. At least SQLite has a high quality reference implementation.

3) It's a shame that SQLite doesn't have a standard of some sort.

</comment></comment></comment></comment></comment></comment><comment><USER6> Cool fact you might not know: SQLite has direct support for "SQLite archives" (i.e. an SQLite database containing a conventional table of {file metadata, BLOB data} records representing archived files) built into its CLI client. This functionality lives under the `sqlite ar` subcommand, which exposes similar command-line functionality to tar(1).

Read more at: [https://www.sqlite.org/sqlar.html](https://www.sqlite.org/sqlar.html)

The nice thing about this is that, since this "sqlar" table is just one table in the file, and the commands work whether or not other such tables exist, a file can be an "SQLite archive" while also being a regular SQLite database containing other tables at the same time. It's sort of like how Fireworks used to add special chunks to PNG—the file was still a PNG, but now it was _also_ a Fireworks project. (But, in this case, the "chunks" aren't opaque binaries, but rather are SQL tables you can manipulate using regular SQL queries.)

And, of course, the SQLar "standard" isn't all that complex—it's designed so that you can easily construct your own SQLite archive files just by issuing regular DDL+DML queries through the SQLite binding of your language of choice. (Though, if you want support for inserting files compressed—which the `sqlite ar` command supports extracting—you'll need a zlib binding as well.)

</comment><comment><USER7> I have migrated a binary file format to an SQLite-based ORM schema at one of my jobs. It worked out amazingly well, with (limited) forward and backward compatibility for nearly free.

A lot of hairy if-else code was made unnecessary.

SQLite is IMHO the best option for data serialization/persistence/interchange in many cases, especially when some data is a blob, or, say, a huge array of floating-point numbers that you need to store with full precision (which makes text-based formats clunky to deal with).

</comment><comment><USER8> > Single-File Documents. An SQLite database is contained in a single file, > which is easily copied or moved or attached. The "document" metaphor is > preserved.

Is this still true by default? With write-ahead (wal) and shared memory (shm) enabled, it is no longer safe to consider only the database file. Pragmas exist to disable those, but people should be familiar of these features and use with care. The document should be updated to introduce these concepts.

<comment><USER9> Why would you realistically need wal or shm for an application file format anyway? That's for high concurrency in general.

<comment><USER10> The article explicitly mentions concurrent access.

</comment></comment></comment><comment><USER11> Sketch ([https://sketchapp.com](https://sketchapp.com)) actually used a zipped SQLite database as it's file format until the past year.

<comment><USER12> Any idea why they changed?

<comment><USER11> They changed to a zip of JSON files, I believe the main reason was likely hackability – one of the main advantages Sketch has in the design ecosystem is it's plugin architecture and the array of tools that exist around it.

If you make your file format easy to work with then it only widens this moat.

</comment></comment></comment><comment><USER13> CoreData has an option to write to a SQLite database, so a lot of data on iOS and macOS apps ends up being written to disk this way.

<comment><USER14> If you code in c/c++, and have even a modest OS/RTOS available, you should use SQLite as your persistence layer, full stop. I switched to this methodology 5 years ago & never looked back. You know what happens when another engineer gets your on disk format & asks how to look at it? You say "open it in SQLite browser"; all further questions disappear.

<comment><USER7> Seconding this.

One thing though: last time I did that, I ended up writing my own ORM/serialization library (with templates and all), but I wish I didn't have to.

Is there anything freely available out there that would allow one to store object hierarchies with the ease of boost::serialization, but into an SQLite EAV scheme?

<comment><USER14> I always _trace_ rather than _log_. That means I have one table per 'tracing function', and just record the arguments to each table. Recording an object means recording the function calls that generated the object; regenerating the object is just playing those functions back. I'll admit that this works far better in C than C++, as C++ tends to treat new types as opaque primitives.

<comment><USER15> Sounds like [https://martinfowler.com/eaaDev/EventSourcing.html](https://martinfowler.com/eaaDev/EventSourcing.html)

</comment></comment></comment></comment></comment><comment><USER16> I believe ZIP supports updates about as well as SQLite. You append the new file to the end (and write the "header" after that). This leaves a hole in the middle, but that can be reused by a later addition. There's some details about internal page size and fragmentation, but I think it could amortize to about the same. There's something to be said about quality of implementation, but that's not an argument about the file format per se.

<comment><USER17> How? There's a massive difference in usability and performance with a SQL database and all the data querying and manipulation it provides compared to a ZIP file that just straps together a bunch of compressed payloads with a key/value file structure.

The linked page even has a case study of the OpenDocument file format that shows exactly how SQLite is better than a Zip file: [https://www.sqlite.org/affcase1.html](https://www.sqlite.org/affcase1.html)

<comment><USER16> If you're using SQLite as a pile of files, you're not doing much querying if the compressed data inside.

> Newer machines are faster, but it is still bothersome that changing a single > character in a 50 megabyte presentation causes one to burn through 50 > megabytes of the finite write life on the SSD.

That's funny, since Firefox would burn through gigabytes of writing to SQLite. All comes down to quality of implementation.

<comment><USER17> The point is that you don't just use it as a pile-of-files when you have SQL access to the data and instead use a high-level data model. This is not possible at all with ZIP.

</comment></comment></comment></comment><comment><USER18> When using kubernetes, I always wonder if we can transform all the tedious "kubeconfig"s into a collection that can be easily queried, modified, saved and retrieved, rather than maintinaing small text pieces in some directories. More ambitiously, maybe include container/image metadata as well?

After reading this document, well, I think I will give it a try. It might not be a perfect fit, but I don't quite like yamls.

<comment><USER19> Lxd uses distributed sqlite for it's config: [https://github.com/lxc/lxd/blob/master/doc/database.md](https://github.com/lxc/lxd/blob/master/doc/database.md)

And there's this simplified fork of K8S that replaces etcd with sqlite: [https://github.com/ibuildthecloud/k3s/blob/master/README.md](https://github.com/ibuildthecloud/k3s/blob/master/README.md)

<comment><USER20> It should be noted that the distributed SQLite is a patchset written by the LXD folks. Last time I talked to hem they said they're working on upstreaming it but want to be sure that everything is sane (SQLite has crazy backwards- compatibility requirements, not to mention the testing).

</comment></comment></comment><comment><USER21> Yeah, I have always felt this is a great idea for almost anything that is not really so small and simple that plain text would make more sense. BLOB-by- nature data like multimedia, pictures etc come as the only reasonable exception.

<comment><USER22> The third great file format (after text and SQLite) is the zip file with custom suffix. That is ideal for BLOB-by-nature data.

<comment><USER21> I didn't really study the formats myself but as far as I know 7z container format is a way better than zip.

<comment><USER22> Very likely true however zip has somehow become a de-facto go-to standard for this stuff. From Java (JAR files) to Microsoft (.docx et al) and numerous open formats including ODF and EPUB.

</comment></comment></comment></comment><comment><USER23> Using sqlite as our format has worked well for us at dannea. It also means we can have web only documents since sqlite can be embedded in a web page

<comment><USER24> > since sqlite can be embedded in a web page

howso ? ... beyond querying the database server-side ...

<comment><USER25> Perhaps they're using SQLite compiled to JS via Emscripten: [https://github.com/kripken/sql.js/](https://github.com/kripken/sql.js/)

<comment><USER23> Yes that is correct, at Dannea we use sqlite compiled to JS. We can even load the data from the same Html page so it allows us to have sqlite powered web apps which can be emailed to others and is true server less

</comment></comment></comment></comment><comment><USER26> I've used SQLite for a distributed project lately where different offline teachers in rural schools would back up their app data to a thumb drive as file databases, take it back to a town with connectivity and upload that data to the app server, so they'd get extra functionality and updates. It was incredibly straightforward to do and worked wonders!

</comment><comment> Const-me It's a good idea unless you're exchanging these files. I'd love couple of my clients switch from HDF5 to SQLite, HDF5 is horrible..

But if you exchange them, it's too easy for app developers to misuse the library causing user's data to be leaked to third parties, in free DB pages, and even in the way how exactly the DB is fragmented.

<comment><USER7> What you suggest sounds like security by obscurity.

<comment><USER27> Based on the mention of free DB pages, I suspect the issue is information leaking because it isn't scrubbed after being deleted, so that it's still in the db file when it's sent off in a third party, just not "accessible" via SQLite APIs. For example, a redaction procedure which consists of deleting one table that contains all the sensitive data doesn't necessarily _erase the data_; you'd probably want to VACUUM or reconstruct the database from a dump.

Moreover, it's a general issue with file formats that can encode the same data in multiple ways that, in addition to the data itself, the data file also incidentally encodes information about the process by which the data was encoded. This metadata is usually ignored and abstracted out by APIs for working with the data, so application developers tend to overlook it as a place where sensitive data can leak.

For comparison, you might consider fingerprinting API clients based on the order in which they send HTTP headers or keys in JSON objects, which can often be correlated with language or library versions in environments where the most convenient map structure is an arbitrarily ordered hash.

It's not necessarily easy to think of nefarious uses for this sort of information, but SQLite database fragmentation can reveal, albeit in rather rough detail, the order in and frequency with which changes were made.

Quite the opposite of "security by obscurity", in fact.

<comment><USER28> The SECURE_DELETE pragma sounds like it'd be useful for this:

[https://sqlite.org/pragma.html#pragma_secure_delete](https://sqlite.org/pragma.html#pragma_secure_delete)

Needs to be manually enabled though, as that doc says it's generally not enabled by default. :/

<comment> Const-me This indeed helps but only partially. Will still leak data in the way the DB is fragmented.

E.g. if you have a text document with 1 row per paragraph, fragmentation will reveal where it was edited, and what kinds or edits were made. You insert a bunch of images into the document, and the physical order of pages will reveal in which order they were inserted, even if you re-arrange them afterwards.

It's possible to workaround by careful programming, but still, very easy to screw up and leak data.

OTOH, with zipped XMLs people normally use instead, you overwrite the complete file each time, it's not fragmented and hopefully doesn't contain any extra info.

</comment></comment></comment></comment></comment><comment><USER29> For those wishing to see Sqlite used to publish data sets:

Datasette ([https://github.com/simonw/datasette](https://github.com/simonw/datasette)) is a new tool to publish data on the web. It uses SQLite under the hood.

</comment><comment><USER30> Why not a text serialization format? That SQLite can read/write.

e.g. tables and schema in JSON.

A table as a list of row-objects, of attributes. (or more efficiently, an object of lists. Or, following the achema, just a list of rows, each a list of attribue's values).

</comment><comment><USER31> I was thinking of using SQLite for creating game saves. Does anyone have experience with that? The more I think about it the more I wonder why we don't see SQLite in games more often.

<comment><USER28> It has been used for game saves before. Not sure how commonly, but we (sqlitebrowser.org) do have links to our website from various places giving tips on how to change things (in the saves). ;)

</comment></comment><comment><USER32> [https://news.ycombinator.com/item?id=18685296](https://news.ycombinator.com/item?id=18685296)

<comment><USER33> I'm surprised this is the only comment in the thread that mentions this security issue. It's only a week old!

</comment></comment><comment><USER34> I have done this on a large .NET Win32/XAML client / server application in two ways:

1) Saving application state and config/settings (window layout, ... very easy to marshall XAML stage to an SQLite file) 2) Caching of common data sets to avoid unnecessary polling to the server of data that is accessed all the time (ie customer master data)

It works beautifully.

</comment><comment><USER35> What if I need to store binary data like images or PDFs? Is storing them as blobs an acceptable method?

<comment><USER36> In SQLite, a database file can be 140 terabytes, [https://www.sqlite.org/limits.html](https://www.sqlite.org/limits.html)

And reading and writing can be even faster than if they were stored directly in the filesystem, [https://www.sqlite.org/fasterthanfs.html](https://www.sqlite.org/fasterthanfs.html)

<comment><USER21> Why don't we replace entire filesystems with SQLite then?

<comment><USER4> > _" This file contains some example code demonstrating how the SQLite vfs > feature can be used to have SQLite operate directly on an embedded media, > without using an intermediate file system."_

[https://www.sqlite.org/src/doc/trunk/src/test_onefile.c](https://www.sqlite.org/src/doc/trunk/src/test_onefile.c)

[https://www.sqlite.org/vfs.html](https://www.sqlite.org/vfs.html)

IIRC, during one of his talks D. Richard Hipp mentions this actually being done for real.

</comment></comment></comment></comment><comment><USER37> You can't stream SQLite files.

<comment><USER38> I'm not sure what you mean by "stream" here. However, there are two APIs designed to facilitate sending SQLite database files over a socket. The sqlite3_serialize() interface ([https://www.sqlite.org/c3ref/serialize.html](https://www.sqlite.org/c3ref/serialize.html)) takes a database and converts it into a blob of contiguous memory that can be sent over a wire. The sqlite3_deserialize() interface ([https://www.sqlite.org/c3ref/deserialize.html](https://www.sqlite.org/c3ref/deserialize.html)) does the opposite, accepting a blob and using it as a database file. These interfaces were created for the purpose of "streaming" database files across network connections. Depending on your specific needs, they might work for you.

<comment><USER39> I'm assuming they mean some parts in the file require parts that appear farther on, so you'd have to receive the whole file?

</comment></comment></comment><comment><USER40> Yaml is stupid. JSON is fine, because it could be `import` from javascript as a real javascript object. In general, make any configuration as a Javascript Object and you can program it whatever you want.

<comment><USER41> 1\. "YAML is stupid" is not constructive. Why is it "stupid"?

2\. One place where YAML is nice is if you want to have a user-edited file. JSON can be a pain to edit by hand due to a lack of comments and being so picky about where commas can and cannot go - YAML isn't perfect, but, it does help with issues like that.

3\. There are languages outside of Javascript, and it's not possible to import a javascript file directly from most of those languages.

4\. There is a difference between the file that you use to configure an app - for which JSON or YAML may be a good fit - and the format for which you export data - where JSON or YAML may or may not be a good fit.

5\. JSON has some very real issues with different parsers not implementing the JSON corner cases the same way - which doesn't really matter, until you hit one of those cases and then it really matters.

6\. JSON doesn't support streaming access to file data - it's not impossible to process a JSON file incrementally, but, it's hard to do as JSON is easiest used when you can load the whole thing into memory.

<comment><USER42> I've been trying to decipher this irrational hatred of YAML coming from a certain subset of developer for years. I've only three or four data points so far, but all of them were very gifted, opinionated developers who were unable to explain their hatred.

Recently, it began to dawn on me that _maybe_ the problem is one of visual acuity required for discerning whitespaces. I always assumed that just because I could visualize a YAML hierarchy just as well as I could JSON, and could edit YAML without messing up the indentation, that others could too. But I've witnessed highly capable developers bang their heads against the keyboard trying to fix repeated parsing errors in YAML they've written. Perhaps not everyone can pick up whitespaces quite so easily...

<comment><USER43> My experience failing with YAML: I've converted a visual workflow tool into a YAML file mode so I could give my users a file format they could edit, share and version with any text editor and commit to a git repo.

The result was that they were at a loss trying to figure out the free-form map and array structures. They could not grasp when to use dashes (arrays) mixed with maps (array of maps) and would fail to indent structures all the time. Trying to create meaningful error reporting was very hard since their mistakes were also valid YAML.

YAML is definitely powerful but can become too complex too fast. And being a format full of sigils and its TIMTOWTDI "there's more than a way to do it" approach can easily enrage lean purists and complexity detractors, hence the hatred.

</comment></comment></comment></comment><comment><USER44> Mh,

> Simplified Application Development

It's depend "application development" is a vast area...

> Single-File Documents

Not much related to SQlite, can be or cannot be done with nearly any format you like. Also remember as a good advise how worse go Catia v6 R2009 (Catia is the best CAD/CAE/CAM suite in the world, used anywhere from automotive to aerospace&defence, it's sole real competitor are Creo (formerly Pro-e) and Nx) when it start to offer single-file PLMs for "light usage"...

> High-Level Query Language

Depending on the format you choose you may have the best/highest level query language: your own apps programming language itself, so IMO may not be an SQLite plus at all

> Accessible Content

Any binary content is FAR LESS accessible than text

> Better Applications

Sorry, it have NO mining at all.

I like SQLite for many aspects but I certainly not look at it as a file format for most applications I use or imaging, for some usage is really nice, for others usages {Tokyo,Kioto}Cabinet is another nice format, for MANY other usage text files in various format, binary files in various formats (including classic pickle) may be better...</comment>


Mathematical Notation – A Guide for Engineers and Scientists (book) - <AUTHOR> https://www.amazon.com/Mathematical-Notation-Guide-Engineers-Scientists/dp/1466230525/ <comment><AUTHOR> Based on this post: [https://news.ycombinator.com/item?id=13951399](https://news.ycombinator.com/item?id=13951399)

...and my comment: [https://news.ycombinator.com/item?id=13951623](https://news.ycombinator.com/item?id=13951623)

...I thought I would just submit the book to root. Highly recommended.</comment>


Charge of the lithium brigade - <AUTHOR> http://www.economist.com/news/technology-quarterly/21651928-lithium-ion-battery-steadily-improving-new-research-aims-turbocharge

<comment><USER2> Article contradicts itself and reality to make lithium ion seem more expensive. First it says that batteries typically cost $500 / kWh, so that a battery for a "small electric car can cost around $10,000". (As an aside, Nissan will sell me a replacement battery for my LEAF for $5499. [1]) This is $229.13 / kWh, because my LEAF has a 24 kWh battery.

Then it says Tesla is selling the 10kWh PowerWall for $3500, which of course is $350 / kWh.

Sloppy reporting. I expect better from the Economist.

1 - [http://www.greencarreports.com/news/1092983_nissan-leaf- batt...](http://www.greencarreports.com/news/1092983_nissan-leaf-battery- cost-5500-for-replacement-with-heat-resistant-chemistry)

<comment><USER3> And of course theres the obvious non-sequitur.

 _Many in the car industry believe the range needs to be close to 500km and the cost around $100 per kilowatt-hour before all-electric vehicles will move into the mass market. That would also allow smartphones and laptops to run for days._

Right, if batteries get cheaper, my laptop will run longer.

</comment></comment><comment><USER4> I really wish that sodium sulfur batteries were seeing more research. They are one of the simplest batteries, and also have one of the highest energy densities. Their main drawback is the need to run at 300 C (575 F). By comparison, an oven bakes between about 175 C to 250 C (350 F to 500 F) and has a cleaning cycle of about 500 C (900 F).

[https://en.wikipedia.org/wiki/Sodium–sulfur_battery](https://en.wikipedia.org/wiki/Sodium–sulfur_battery)

[http://www.kemet.com/Lists/TechnicalArticles/Attachments/121...](http://www.kemet.com/Lists/TechnicalArticles/Attachments/121/2013-11%20Sodium- Sulfur%20Batteries%20and%20Supercapacitors.pdf)

I'm more concerned about the 100 pounds of sulphur in an electric vehicle battery than the temperature. An accident in water could be particularly explosive. Then again, we use things all of the time that contain poisonous or explosive compounds.

So what I think would have tremendous potential (cough) is the mass production of small AA size sodium sulphur batteries. I picture them strung together like a chain for a reel to reel tape player, so only the read/write head would be hot. Play to charge, rewind to discharge, how hard is that?. This is 1960s technology. The only esoteric component is the beta-alumina solid electrolyte (BASE), the rest could be assembled by anybody.

The future is almost certainly high density batteries with supercapacitor assist. Sodium sulphur batteries could be the bridge until Moore's Law of lithium batteries makes them affordable.

P.S. Just or fun: the nickel-iron battery:

[https://en.wikipedia.org/wiki/Nickel–iron_battery](https://en.wikipedia.org/wiki/Nickel–iron_battery)

These should have long ago replaced lead-acid batteries (which are probably only used due to their short lifetime which promotes a thriving production infrastructure, much like razor blades).

<comment><USER5> No shit! I was super excited about sodium-sulfur about six years ago when Ceramatec developed a ceramic membrane claimed to allow low-temperature NaS batteries (IIRC, below 98 degrees C, with solid Na electrolyte). I've heard nary a word since.

</comment></comment><comment><USER6> Shouldn't it be this link ? [http://www.economist.com/news/technology- quarterly/21651928-...](http://www.economist.com/news/technology- quarterly/21651928-lithium-ion-battery-steadily-improving-new-research-aims- turbocharge)</comment>


PHP Dark Arts: Daemonizing a Process - <AUTHOR> http://www.re-cycledair.com/php-dark-arts-daemonizing-a-process

<comment><USER2> Is there any advantage to this v.s. using <http://pear.php.net/package/System_Daemon> ?</comment>


Ask HN: Secure VPN unlimited - <AUTHOR>

Do you use VPNs to access Internet over public Wi-Fi?<p>Any suggestions for companies you're happy with? <comment><USER2> Came across [https://www.tunnelbear.com/](https://www.tunnelbear.com/) the other day. They seem pretty cool.

I've used [https://www.privateinternetaccess.com/](https://www.privateinternetaccess.com/) while traveling abroad. It has worked quite well.

</comment><comment><USER3> [https://cryptostorm.is/](https://cryptostorm.is/)

Varying levels of anonymity in purchasing, accepts cryptocurrencies, and compatible across most of my devices. I've also had some great customer support w/ them.

</comment><comment><USER4> [http://vpnbook.com/](http://vpnbook.com/)

Offers TCP 443, UDP 53 and PPTP.

</comment><comment><USER5> get a digital ocean instance and install openvpn. periodically destroy the instance and rebuild it.

<comment><USER6> wont that instance data be stored in a backup a DO?

<comment><USER7> The point of destroying is presumably that you don't pay whilst you're sleeping, or otherwise not using the server.

Re: data, you can (optionally) do a wipe when you destroy a DO, but you could just delete the logs or turn off logging, which would be faster. Having said that, I'm not sure what in the logs would be harmful to you.</comment></comment></comment>


Surface disk space FAQ - <AUTHOR> http://www.microsoft.com/Surface/en-US/support/surface-with-windows-RT/files-folders-and-online-storage/surface-disk-space-faq

<comment><USER2> This is the same B.S. that happened with the Xbox 360 hard drives.

In 2006 Microsoft sold a 20GB hard drive for $120 (which could have bought at least a non-branded 100 GB hard drive)

When you used it with the 360, the hard drive would only report 14 GB because 6GB were used for the OS.

<http://support.microsoft.com/kb/911593>

FYI Apple's operating system takes up 1GB, so this is relevant because a 32GB Surface has a much usable storage as a 16GB iPad (source: [http://www.businessinsider.com/microsoft-surface-storage- siz...](http://www.businessinsider.com/microsoft-surface-storage- size-2012-10))

<comment><USER3> > _6GB were used for the OS._

Not really the same though, since the XBox OS fully resides in Flash/ROM/however-you-want-to—call-it-internal-storage 16MB ( _megabytes!_ ) filesystem, and you can readily boot without the hard disk[0]. Those 6GB were merely "set aside".

The price is a complete rip-off though.

[0] this may have changed since NXE, but the OS is still extremely small by today's standards, e.g the last one is 113MB: _[http://support.xbox.com/en- US/xbox-360/system-updates/system...](http://support.xbox.com/en- US/xbox-360/system-updates/system-updates- info#00569e16d2844366beafb52b4d747a77*)

<comment><USER4> As of November 19, 2008 the Xbox OS actually requires additional disk space. It's why they discontinued the Xbox 360 Arcade console package since it didn't come with a hard drive.

"While previous system updates have been stored on internal memory, the NXE was the first to require a storage device—at least a 128 MB memory card or a hard drive." <http://en.wikipedia.org/wiki/Xbox_360#New_Xbox_Experience>

</comment></comment></comment><comment><USER5> I must say though, they do get a lot of bonus points for honestly laying out exactly where everything went and how much space each component takes up. On top of that, they even gave storage improvement options aren't simply giant advertisement for their cloud services. Color me impressed with MS's upfront honesty.

<comment><AUTHOR> Not exactly "upfront honesty" when they didn't say this before they started advertising as a "32 GB tablet" knowing _full well_ that what consumers will think is that they get 32 GB of storage for a $500 price.

They only did this now after others have already debunked their storage claim:

[http://semiaccurate.com/2012/10/31/microsoft-surface-can- not...](http://semiaccurate.com/2012/10/31/microsoft-surface-can-not-compete- against-real-tablet/)

Also shame on most tech reviewers who didn't even notice this in their reviews, or didn't want to say it.

<comment><USER6> I think you have unrealistic expectations. If you read an advertisement for a computer with a 250GB hard drive, do you really expect it all to be available space? I don't think anyone else does, so why would they think that in this case?

<comment><USER7> I would expect the vast majority of it to be free space.

If you bought a laptop with a 250GB hard drive, and found when you booted it that ~125GB were used by the OS, would you be okay with that? Especially when the HD storage was a prominently advertised feature?

</comment></comment></comment></comment><comment><USER8> So when Microsoft were talking about having a consistent experience between tablet and PC, I didn't think this was what they'd be thinking of. This reminds me of my current desktop, which I ran out of space on the SSD almost immediately and then spent an instructive half-hour cutting down swap space and getting rid of the hibernation file. Seems like the first thing to do on a Surface would be to find some way of freeing up that space.

I feel a bit mean poking fun at Microsoft now, but honestly, it amazes me that they'd think it was acceptable to take up half the hard drive of your tablet - especially given that storage space is one of the major metrics that mobile devices are sold on at the moment!

<comment><USER9> The microSD slot allows you to add up to 128GB of storage fairly cheaply to your Surface.

I don't see this as a big problem.

</comment></comment><comment><USER10> Really?

Recovery tools on a TABLET?

What's the size of iPad firmware again? Around 1GB apparently

How much is the crud that Windows accumulated all this time?

I'm beginning to think all the 16-bit pieces of windows are still there (I mean, the part that compiled)

Talk about a monolithic mess

<comment><USER11> This sort of size is necessary to support a tablet which isn't hopelessly tethered to a PC. The iPad avoids a big recovery image by requiring you to restore it from your PC. You've offloaded this function onto an (arguably) more capable device. But how much external storage can you buy for the cost of a whole PC. For some users who don't create a lot of content, the Surface realistically could be their only device, and being able to restore it without a PC is a lot of convenience.

<comment><USER12> This is no longer the case with iCloud.

All of my 'iDevices' are now untethered from my computer. When I bought my iPhone 5 they actually made me wait for the iCloud restore to complete before I could leave the store.

Backed up my iPhone 4 before jumping the train to the Apple store. Walked in, purchased the new one, iCloud sync, walked out with virtually the same data I walked in with on a faster and prettier device =)

<comment><USER13> Is this being touted as an Apple feature that Android retroactively stole 3 years in the past yet?

<comment><USER14> Downvoted. Nobody mentioned anything about Apple being first, merely that this feature exists and is useful.

</comment></comment></comment></comment></comment><comment><USER15> What a fantastic time to live in, where we can describe 128 billion things with the adjective "only", and sniff derisively at them.

<comment><USER16> Well if you wanna go that route, a loaf of bread for dinner is 'only' trillions and trillions of atoms.

<comment><USER17> And a lot of those atoms are just air!

<comment><USER18> No they're not, they're empty space.

<comment><USER7> "Empty space" isn't made out of atoms. And yes, there's plenty of air in a loaf of bread.

</comment></comment></comment></comment></comment><comment><USER19> Then it's a good thing Class 10 MicroSD cards are under $1/gig?

<comment><USER20> Yup.

Kinda felt like Google shot themselves in the foot by missing them out of their nexus tablets. The idea that it's a negative because people don't like managing what they use the space for... well it doesn't fly with me.

<comment><USER21> Yeah - they should at least have included read-only SD card reading, for media consumption. My dad currently does not know what tablet to buy because reportedly the Google devices are great, but they don't have the slot.

</comment></comment></comment><comment><USER22> Well, at least you can plug a card in there.

Just out of curiosity, how is it on other tablets (ios, android). Is some portion of the marketed memory occupied at all times? If so how big is the percentage?

<comment><AUTHOR> iOS and Androd only take 1 GB or less, and there's another 8% or so that comes from the conversion of 160,000 MB to 16 GB.

<comment><USER23> That tis not correct. It's much more than a gig.

</comment></comment></comment><comment><USER24> Well at least they come right out with it and explain _exactly_ why the free space is what it is.

</comment><comment><USER25> While it's not news for a computing device to rely on ignorance in regards to marketing of specs, at what point does it cross the line from casually misleading to intentionally deceptive?

</comment><comment><USER26> I know humor is frowned upon here, but it makes me think of the following: <http://i.imgur.com/Rcu0C.jpg>

</comment><comment><USER27> Yes, but you can stick a 32 or 64GB SD card in it(or swap through many as needed) because it has an SD card slot. You can even put in USB flash drives since it has a full size powered USB port with hub support. Hell,you could connect a 2TB portable USB drive and it would work(at the expense of battery). Try doing that with an iPad or Nexus 7 or even the Nexus 10. Google's devices are supposed to be more open but I cannot understand why they want to force upsell you to the more expensive options to add additional storage by omitting MicroSD slots.

<comment><AUTHOR> The point still remains that they advertised the tablet as being higher value, when it's not. You can get a Nexus 10 with 16 GB of storage and a much higher resolution than the Surface for $400.

<comment><USER28> The 'value' you speak of in tablets is not in the volume of space available but in the overall experience. I can't say I've ever heard a "real user" express how they wished they have more physical space on their tablet or smartphone. It's a nice line item on a spec sheet to hold over your competitors but how many users actually care?

<comment><USER29> You have never heard a "real user" (and remember, you and I are "real users" as well; you don't stop being a real user just because you are technically inclined) complain about having run out of space and having to clear off space? About not being able to fit all of their music, photos, videos onto their MP3 player, phone, or tablet?

I've certainly encountered that. And I can imagine people saying "hey, this has 32 GB of space, my current device only has 16, this will allow me to fit twice as much", without realizing that a full _half_ of that space is not actually available.

<comment><USER30> You can easily drop in a SDXC card in the Surface if you need additional storage, unlike the other main alternatives like the iPad and Nexus tablets. You can load up any number of movies to take on a trip on microSD cards and swap them in and out. What do you do on the other tablets without the Microsd slot when you run out of storage?</comment></comment></comment></comment></comment>


The power of CAD meets the simplicity of paper - <AUTHOR> https://www.plm.automation.siemens.com/labs/catchbook/

<comment><USER2> Ivan Sutherland is calling from 1964 and wants his app back...Sketchpad [https://youtu.be/USyoT_Ha_bA?t=269](https://youtu.be/USyoT_Ha_bA?t=269) [video]</comment>


Microsoft's Surface Not Suitable for Commercial Use - <AUTHOR> http://unwiredben.livejournal.com/305479.html

<comment><USER2> That's an interesting catch.</comment>


SpaceX FCC Application for over 4000 Internet Satellites - <AUTHOR> http://licensing.fcc.gov/cgi-bin/ws.exe/prod/ib/forms/reports/swr031b.hts?q_set=V_SITE_ANTENNA_FREQ.file_numberC/File+Number/%3D/SATLOA2016111500118&prepare=&column=V_SITE_ANTENNA_FREQ.file_numberC/File+Number <comment><USER2> You gotta love the genius of a guy whose rocket company gets to practice for their Mars launch by sending up a bunch of satellites (powered by the solar panels he makes) to assemble a new worldwide wireless internet company, in order to provide ubiquitous connectivity for his autonomous car business.

<comment><USER3> Henry J. Kaiser also did it all. Kaiser Steel. Kaiser Motors. Kaiser Permanente (the cement company, not the hospital company). Kaiser Shipbuilding. Kaiser Gypsum. All gone now.

<comment><USER4> While the entities might be transient, the progress they deliver is permanent.

Musk might not care if Tesla and SpaceX disappear if electric vehicle mobility and affordable transport to Mars is achieved. They are a means to an end.

<comment><USER5> I'm sure he cares quite deeply about the legacy of his companies, but his core goals, as stated, transcend the current organizations.

He has said multiple times that he thought the likelihood of success for each of Tesla and SpaceX was less than 10 percent, but that was ok, because even in failure, they would 'move the ball forward' on sustainable energy and making humanity a multi-planetary species.

</comment></comment></comment></comment><comment><USER6> So from the Reddit discussion it seems like 200 launches is about a fair guess for the total deployment of this system. It seems like a perfect way to derisk the notion of reusable launch systems while providing a native-to-SpaceX revenue stream (orbital internet provision).

Assuming that they can achieve the success rates they've been aiming for, demonstrating lowered risk and cost through their own deployments will provide about the best advertisement you could possibly imagine. Waiting for organic outside demand to fill the launch roster to the same point would take time that Musk clearly does not want to waste. It seems to be a Keynesian approach to the slightly Chicken/Egg problem of commercial space.

<comment><USER7> Isn't the launch roster way beyond full already? [http://www.spacex.com/missions](http://www.spacex.com/missions)

<comment><USER6> I don't mean that it's full from a financial perspective, but full in an argument-ending "this is the way of the future" sense.

Edit: There are ≈80 missions listed on the link there, 200 launches for their own missions would provide ample opportunity to derisk their customers by reusing launch vehicles beyond their currently marketable mileage and proving their safety (I can imagine that one would want some pretty steep discounts for a rocket that's been used 5 times given current success rates).

<comment><USER8> Also _im_ -proving the safety by getting tons of experience on what works and what breaks, while only risking some of your own hardware.

<comment><USER6> I definitely agree with you on this point. More launches means less failures per n launches.

</comment></comment></comment></comment></comment><comment><USER9> Coming soon: Elon Prime

Pay 40k a year and get unlimited SpaceX Internet, a Tesla w/ free charging, and a complimentary flight/weekend stay at Solar City Space Hotel.

Who's in?

<comment><USER10> Honestly, I don't think I'd blink an eye at signing up for that. Space hotel trip alone is worth that!

In fact, that makes me seriously think about the ITS for earth orbit activities. Add a nice big docking port to connect it to a space station, and you've got yourself a relatively cheap reusable vehicle to fly a whole mess of people into LEO.

<comment><USER11> Virgin Galactic is charging 250k for their "some minutes in weightlessness" ride...

<comment><USER12> Clearly thats a a stepping onto something better... isn't it.

I mean the planes in the beginning could charge high rates for just flying someone in a circle, but today planes need to be fast, reliable, on time all while being cheap enough that most Americans can afford them.

</comment></comment></comment></comment><comment><USER13> I'm late to this thread and very disappointed in the HN crowd that nobody has mentioned any of these things.

OneWeb is trying to do the same thing and already has rights to the spectrum from the ITU. They are using Arianespace to launch and Airbus build their spacecraft. It's possible this whole application is a ploy to disrupt Arianespace since SpaceX is upset about not getting a massive launch contract.

[http://spacenews.com/one-year-after-kickoff-oneweb-says- its-...](http://spacenews.com/one-year-after-kickoff-oneweb-says- its-700-satellite-constellation-is-on-schedule/)

Commercial satellite constellations do not have a great track record. Iridium and GlobalStar tried to do phone networks and both went bankrupt. GlobalStar was backed by Qualcomm and Iridium by Motorola. Though they both have complete, functioning constellations. Look at GSAT and IRDM to see how well they are doing post bankruptcy.

Teledesic tried to do the same thing in the 90s and was backed by Bill Gates. It didn't go anywhere.

<comment><USER14> Tren Griffin, the 4th employee at Teledesic, has a good write up of it with lessons learned. May be able to see if anything has changed that may make these new efforts work where others failed.

[https://25iq.com/2016/07/23/a-dozen-things-i-learned- being-i...](https://25iq.com/2016/07/23/a-dozen-things-i-learned-being- involved-in-one-of-the-most-ambitious-startups-ever-conceived-teledesic/)

<comment><USER15> "You probably have heard the story about the chicken being involved in breakfast because she laid an egg but that the pig was committed to breakfast since he supplied the bacon. Investors are like the chicken, but startup up employees are like the pig." I liked this quote... perfect way to describe the startup-investor relationship.

</comment></comment></comment><comment><USER16> My bet is that the "pizza box" phased array rooftop antenna will evolve into a complete base-station. We know it has a multi-element RF front-end feeding in to massively parallel software defined radio (SDR). Likely available in PoE or wireless variants (solar+batteries+wifi; imagine the MIMO with 100 antenna elements!).

SDR is quite interesting because it can track moving satellites, and do simultaneous multipath reconstruction of multiple terrestrial broadcast signals -- digital tlevision, FM/AM/satellite radio, shortwave, etc. No antenna pointing needed, as it's all done in software. After demodulating the carrier the video and audio content is streamed on a local web page available to computers/phones/tablets on the LAN. Existing TVs can be supported via an HDMI or analog (VGA/composite/3.5mm) stick w an RF remote.

Optional (or one-time software unlock?) on-board storage for multimedia time- shifting and distributed CDN caching (reducing bandwidth and latency by keeping commonly served duplicate files like js libraries locally). Having a local server also lets them do fun things in the future like transparent LZMA compression, optional ad/telemetry/malware blocking, data pre-caching, smart shared HTTP caching, etc.

The inclusion of on-board storage allows data to be sloshed around to best utilize the terrestrial spectrum (broadcast content) and satellite spectrum (multicast content). It's much like Tesla batteries allow energy to be sloshed around to best utilize distributed solar and the utility grid.

With proper RF front-end design, support for the diversity of global broadcast standards can added through OTA software updates via satellite. This simplifies hardware into a single "world" model with regional software variants.

Easy install, especially with the wireless version (mount on pole, erect, securely fasten, done). Broadcast signal reach and convenience (any device or TV) that beats the pants off of any OTA antenna out there. Convenient access to terrestrial broadcast content w time-shifting takes some of the load from streaming services, so it's a strategic move and not just a "gee wiz" feature they got cheap because of SDR.

At $200-300 for the equipment and $50/mo for internet, with a good multimedia time shifting UI, SpaceX would _cream_ every rural data provider out there.

Anyway, just the idle dreams of an internet user living in a rural satellite "data desert"...

<comment><USER17> I've worked with phased array flat panel antennas used for geostationary Ku and Ka band two way satellite access.

Phased array flat panel antennas cannot be purely software steered sufficiently to track a low earth orbit satellite from horizon to horizon. It's not good enough to just put a flat panel phased array facing straight up, you won't have enough gain aimed in the direction where the satellite actually is at any given moment. Assuming a CPE device that measures 1.5'x1.5' to 2'x2' in size, it will still need a motorized azimuth and elevation stepper motor system (or equivalent) to track a LEO satellite.

If you're trying to do multi-Mbps data to/from a satellite in bands >10GHz (and probably >18GHz), the path loss through the atmosphere and RF parameters mean that you want as much gain as possible... This is why the most economical solutions for high capacity two way satellite through geostationary transponder capacity involve 1.8 to 3.0 meter size dish antennas. Gain is important. Gain and enough signal over noise ratio that you can do higher order QAMs and not spend half your bitrate on FEC.

You could, I suppose, if the antenna system was cheap enough and large enough build a non moving, cone or pyramid shaped antenna covered in phase array segments that could maintain a high bandwidth connection in the >10GHz bands to a moving LEO satellite. But it would be pretty large.

edit: for those who are curious and want to see an example of a parabolic (steered) use for moving satellite comms:

[https://www.google.com/search?q=melco+antenna+mitsubishi&ie=...](https://www.google.com/search?q=melco+antenna+mitsubishi&ie=utf-8&oe=utf-8&client=firefox-b)

[http://proceedings.kaconf.org/papers/2015/bsw_4.pdf](http://proceedings.kaconf.org/papers/2015/bsw_4.pdf)

there is also an israeli company which is a competitor to mitsubishi in this space. and a few others.

if you watch the weekly FCC filings for new satellite equipment licenses, there's a lot of MELCO antennas flying around on top of mid and large sized jet aircraft.

<comment><USER16> Thanks, great comment.

Mark my words, _SpaceX is quietly working toward nothing less than a revolution in global data infrastructure_ , just as Tesla is working to accelerate a revolution in global energy infrastructure. Sneaking in some software, onboard storage, putting them on all Tesla cars, mesh networking the terrestrial base stations with MIMO links, etc.

Cars suddenly become a global peer-to-peer mesh sneakernet. It would be fun if Tesla distributed data via "pulses" between passing cars... (signed high resolution maps? deep learning connectomes? the latest Netflix original series? :D)

>Phased array flat panel antennas cannot be purely software steered sufficiently to track a low earth orbit satellite from horizon to horizon.

Righto. SpaceX plans to steer the beam down to 40° from the horizon. This also minimized path losses.

Drawing from the SpaceX filing: [http://i.imgur.com/7zOZ7kr.png](http://i.imgur.com/7zOZ7kr.png)

>All Ku-band downlink spot beams on each SpaceX satellite are independently steerable over the full field of view of the Earth. However, user terminals at the customers' premises communicate only with satellites at an elevation angle of at least 40 degrees. Consequently, as shown in Figure A.3.1-1 below, each satellite operating at an altitude of 1,150 km will provide service only up to 40.46 degrees away from boresight (nadir), covering an area of about 3.5 million square kilometers (1,060 km radius)

source: Figure A.3.1-1 on pp6 of the FCC technical information document, currently 404, via /u/SkywayCheerios

[https://www.reddit.com/r/spacex/comments/5d9724/spacex_has_f...](https://www.reddit.com/r/spacex/comments/5d9724/spacex_has_filed_for_their_massive_constellation/da2qjj7/)

[https://licensing.fcc.gov/myibfs/download.do?attachment_key=...](https://licensing.fcc.gov/myibfs/download.do?attachment_key=1158350)

[http://webcache.googleusercontent.com/search?q=cache%3Alicen...](http://webcache.googleusercontent.com/search?q=cache%3Alicensing.fcc.gov%2Fmyibfs%2Fdownload.do%3Fattachment_key%3D1158350)

<comment><USER17> > SpaceX is quietly working toward nothing less than a revolution in global > data infrastructure

I mean, that's a really cool 40,000 ft view idea and all, but global data infrastructure is made up of things like the Hibernia Atlantic cable, its several dozen cousins of post-2000 transatlantic and transpacific cables with DWDM terminals on both ends, and cool things like 100, 200, 400Gbps per wavelength coherent QPSK/16QAM DWDM terminals. And major terrestrial traffic exchange points of existing Internet infrastructure where ISPs put $150,000 core routers (example: 60 Hudson, NAP of the Americas, Telehouse Docklands, Otemachi Building in Tokyo, One Wilshire, 350 E. Cermak, etc).

Satellite traffic is a pretty tiny drop in the bucket compared to terrestrial backbone infrastructure. It's a truly admirable goal to bring affordable broadband to really impossible to reach locations via satellite. On the other hand there are a lot of emerging terrestrial WISP technologies and PTP microwave technologies that can be used in rural areas to provide bandwidth without the need to pipe it up into space and back. It's a lot cheaper to establish a tower site on top of a mountain, even if you need to bulldoze a road to the top, and spend $30,000 on routers and PTP/PTMP radio gear.

In some of the areas of the rural western US where my network engineering job touches, there are WISPs which are rapidly eating into the customer base of people who are (rightfully so) dissatisfied with highly oversubscribed consumer grade VSAT satellite systems.

In the end there will be a combination of many things. The next generation of high capacity Ka-band geostationary satellites (ViaSat-2, etc) are a lot higher capacity. Services like o3b allow ISPs to buy a dedicated 1:1 high capacity pipe to places that can't be reached by PTP microwave and are uneconomical to reach by submarine or terrestrial fiber (example, all of o3b's new pacific island nation state customers). There's traditional geostationary c and ku band capacity from SES, Eutelsat, Intelsat, AsiaSat, russian companies, etc. And of course terrestrial fiber. You don't need a huge amount of money to run singlemode these days, assuming aerial wood poles and a mostly rural area, you need two guys, a bucket truck and about $10,000 worth of tools.

<comment><USER16> Completely agree on WISPs. I think SpaceX will be pursuing mesh networking too.

Could an OTA update (and repointing 90°) turn a satellite terminal into an auto/calibrating point-to-point MIMO link?

>global data infrastructure is made up of things like the Hibernia Atlantic cable

Surprisingly the stated primary goal of the SpaceX constellation is actually to compete with long-distance fiber backhauls. Their sell for satellite backhaul is that it's lower latency (no need to avoid continents, 50% faster speed of light, fewer hops) and works everywhere.

Giving global gigabit internet to rural areas and ~10% of urban customers (with the rest on fiber) is only a bonus. :)

Most of what we know about the plan comes from this video. Timestamp is to the start of the juicy bits. [https://www.youtube.com/watch?v=AHeZHyOnsm4&t=2m10s](https://www.youtube.com/watch?v=AHeZHyOnsm4&t=2m10s)

<comment><USER17> IMHO the shannon limit and basic laws of RF/path loss and channel capacity say that satellite is not a true competitor to backbone links by singlemode fiber. For the equivalent of "last mile" services, yes, but not as a replacement for laying fiber between points A and B.

The entire data throughput capacity of a current generation, 5500 kilogram, geostationary Ka-band satellite that costs $185 million to build and launch is much, much less than the 80 channel x 100 Gbps per channel DWDM system you can run on two strands of 9/125 singlemode fiber. And vastly less than the 144, 288 or 864 strand count fiber cable you would see laid between two cities by a carrier-of-carriers operator like Zayo these days.

It is fabulous to see more competition for high priced monthly-leased transport kHz/MHz from geostationary satellite operators. O3b was an amazing thing (and still is). More competition is good. But it's a pipe dream to say that satellite backhaul will ever be preferable to fiber carrying N x 10GbE circuits or a 100GbE circuit...

[https://en.wikipedia.org/wiki/Shannon%E2%80%93Hartley_theore...](https://en.wikipedia.org/wiki/Shannon%E2%80%93Hartley_theorem)

[https://en.wikipedia.org/wiki/Noisy- channel_coding_theorem](https://en.wikipedia.org/wiki/Noisy- channel_coding_theorem)

<comment><USER18> > IMHO the shannon limit and basic laws of RF/path loss and channel capacity > say that satellite is not a true competitor to backbone links by singlemode > fiber. For the equivalent of "last mile" services, yes, but not as a > replacement for laying fiber between points A and B.

The filing states they'll be using free space optics / lasers between satellites. The Ka/Ku links are only for the initial uplink and downlink.

> But it's a pipe dream to say that satellite backhaul will ever be preferable > to fiber carrying N x 10GbE circuits or a 100GbE circuit...

It is not a pipe dream. Free space optics in... well space, have a 50% propagation latency advantage vs terrestrial fiber. This helps equalize things somewhat.

<comment><USER17> > The Ka/Ku links are only for the initial uplink and downlink.

Then there's a huge bottleneck, if the links from the satellite constellation as a whole to the trunk earth stations (not the CPEs) are high capacity Ka- band, there's RF issues with capacity...

It's like if you have a network that's composed of a whole lot of 10GbE backbone links from router to router and your IP transit connection to upstream ISPs/the global v4/v6 routing table goes through one 1000BaseLX link.

<comment><USER18> No, it's nothing like that. They're talking about highly localized signals via phased arrays on both sides. Additionally you underestimate the capacity of wireless: LTE has no problem doing 30 bits per 1 hz of spectrum. These will run at a lot more than a 1 gbit globally shared last mile.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER19> I'm still not sure why they are doing this - I'm in West Africa now, have been through some _very_ poor countries, and every single one has blazing fast 3G connectivity, even in the remote mountain towns.

I'm tethered to 3G, and for all purposes it's as fast as cellular internet I've used anywhere in the world.

It _is_ a little more expensive than most locals really can afford, although they are all checking facebook multiple times per day.....

So I really don't understand this idea of "bring internet to the 3rd world" they have it already.

<comment><USER20> I'm sure there's a term for this, but people in the West have this idea that everyone not in the West is a dirty savage living in a mud hut.

Also, you aren't a real rich person until you've "helped the Africa".

<comment><USER19> > _people in the West have this idea that everyone not in the West is a dirty > savage living in a mud hut._

Yes, very much. Now I've been moving around for 5 months my eyes are very much open to the reality.

People also thought I would be murdered on day 1, when in fact I have met many, many kind kind friendly people who have welcomed me into their home.

The culture of the West really does teach it's pupils they are the best, and everyone else is crap. The Media further enhances this idea.

I loved when I met a guy in a rural, poor part of Gambia - he said "Oh yeah, my brother lives in Australia. I was there for a few years. I came back here, I like it more".

Africa is not the hell-hole people think it is.

<comment><USER21> Let's not go too far in the other direction though. I live in Kenya and there are certainly some horrific aspects about living here. However, if you make at least a living wage and learn the rules of engagement you can live quite a balanced life

</comment></comment></comment></comment><comment><USER22> The discussion on r/spacex:

[https://www.reddit.com/r/spacex/comments/5d9724/spacex_has_f...](https://www.reddit.com/r/spacex/comments/5d9724/spacex_has_filed_for_their_massive_constellation/)

Interesting note from the Organization section where it shows that Elon's trust owns 54% of outstanding SpaceX shares and over 70% of the voting shares.

<comment><USER4> Interesting for sure, but not unexpected. You would not entrust your life's work to your shareholders without having overwhelming control.

<comment><USER23> Unless I'm mistaken, many founders do give up control to their share holders.

<comment><USER24> Yes, but Elon himself has put $100M into the company, and most of that early. Musk is also popular of the "space travel is scary and there will be accidents; if we put it in the hands of the shareholders it'll get shut down" argument and that's certainly a part of his negotiating position.

</comment></comment></comment></comment><comment><USER25> fcc.gov seems unresponsive, here's a Google webcache link to the HTML application: [http://webcache.googleusercontent.com/search?q=cache:3lStj71...](http://webcache.googleusercontent.com/search?q=cache:3lStj71lnE0J:licensing.fcc.gov/myibfs/download.do%3Fattachment_key%3D1158354+&cd=1&hl=en&ct=clnk&gl=us)

</comment><comment><USER26> So after blowing up Facebook's Internet-providing satellite, they're going after their own constellation.

I do wonder how this will affect potential customers who would launch communications satellites on SpaceX, but are worried about handing over something proprietary to a competitor.

<comment><USER27> They have conclusively demonstrated that relying on a small number of satellites in the small number of launches concentrates too much risk. Unfortunately the demonstrated this accidentally blowing up a satellite.

In the long run though, success at scale with a system like this relies on the typically guaranteed losses. A certain percentage of rockets will fail, a certain percentage of satellites will fail, a robust system will be designed to accommodate this.

(Of course, and this is probably the topic of a totally different thread, the tolerance for this must change dramatically when there are people in there.)

<comment><USER28> there is still the issue of space junk, more satellites, more launches, more mishaps, more junk. And now you have more junk to create more mishaps.

I'm not saying I want more concentration, I'm just saying that the path to de- centralisation is not that clear cut (at there is a threshold were we have to talk about CO2 emissions).

<comment><USER29> Junk prevention is addressed in the details of the proposal, deliberate deorbiting early, but if not they'll decay nicely. It's still as many satellites as are currently flying at all, but they have considered it.

</comment></comment></comment></comment><comment><USER30> I've looked through the reddit post and this and haven't found much discussion regarding the increase in latency that would come with such a service. Does anyone have a general idea of how much slower a round trip is when you count the transmission to the satellites and back? In some multiplayer games a difference of 100ms provides a massive advantage, making it much harder to play competitively on slower connections.

<comment><USER31> The satellites are on Low Earth Orbit (LEO), so they are much closer to Earth than typical communication satellites. When the distance from ground station is 1000-1500 kilometers up, latency is not a problem. In fact, latency via satellites may be smaller than through fiber as speed of light is greater in vacuum. IIRC in fiber it is something around 0.6c, so a 40 % off from a space- based approach. Also there's probably going to be more switches along a ground-based path than along the satellite-to-satellite route.

<comment><USER32> Perfect solution: communicate with neutrinos directly _through the earth_ for a point-to-point absolute minimum latency link.

<comment><USER33> Brilliant. Only problem is the enormous antenna and the >99.9% ping timeout rate :-)

<comment><USER32> Depends upon the signal strength?

</comment></comment></comment></comment></comment><comment><USER34> The PSLV launched 20 satellites in one shot and that vehicle is 15M $ per launch. The double digit million cost of SpaceX's reusable rocket isn't much cheaper than that.

Does someone know what orbit these are being launched into and how heavy they are?

<comment><USER35> You can find launch capacities on Wikipedia. PSLV has a smaller payload and smaller price than F9.

For these satellites, you can find a description of their orbit and a guess at their mass at [https://en.wikipedia.org/wiki/SpaceX_satellite_constellation](https://en.wikipedia.org/wiki/SpaceX_satellite_constellation)

</comment></comment><comment><USER36> Would a hand-held size device be able to connect to these satellites? I'm imagining a mass-manufactured device like a Peek[0] that could be distributed in countries with heavy censorship...

[0][https://en.wikipedia.org/wiki/Peek_(mobile_Internet_device)](https://en.wikipedia.org/wiki/Peek_\(mobile_Internet_device\))

<comment><USER37> The receivers/Earth stations will be phased array antennas; basically an antenna 'grid'. They won't fit in your pocket, but they will fit on top of your car.

Keep in mind, this system is designed to give the Internet to _everyone_. The antennas will be small enough to conceal. Since the signals go up instead of across a border, censorship is not an issue.

<comment><USER38> You need to get landing rights in all countries you plan to operate your frequency bands in. They cannot simply start transmitting down to any country without previous authorization.

<comment><USER39> Unless that particular country possesses anti-satellite weaponry, I think you absolutely can. Iran hasn't had much success censoring satellite communications on their soil.

[http://www.dailymail.co.uk/news/article-3706645/Iran- destroy...](http://www.dailymail.co.uk/news/article-3706645/Iran- destroys-100-000-satellite-dishes-stop-citizens-watching-morally-damaging- television-corrupts-Islamic-values.html)

[https://www.engadget.com/2016/04/22/toosheh-iran-internet- ce...](https://www.engadget.com/2016/04/22/toosheh-iran-internet-censorship/)

[http://www.al-monitor.com/pulse/afp/2016/07/iran- television-...](http://www.al-monitor.com/pulse/afp/2016/07/iran-television- culture-satellite-censorship.html)

<comment><USER28> yeah sure, you can shit on a country's regulations, the response ranges from harmless to starting an open war, with a a few steps like seeking the company's funders extradition, seeking redress in international arbitrage, blocus etc.

<comment><USER40> And the US allows its companies to ignore most of the possible responses.

I imagine Musk could get even more than the already generous US government protection if he puts the base stations on US soil (I.e. all traffic in NSA reach)

</comment></comment></comment></comment></comment></comment><comment><USER41> Currently paying US$60 a month for land based internet in NZ and a friend pays $160 for the crappy satellite equivelent. Several million subscribers would not be unfeasable and provide a pretty massive income stream

<comment><USER42> Add another 8 million or so from Australia

<comment><USER43> Lots of folks outside the cities in America too. I've heard estimates between 10-30 million.

</comment></comment></comment><comment><USER32> Makes me think: by the time we get to Mars, Mars will have Internet (and GPS etc).

</comment><comment><AUTHOR> I wonder if a distributed model would work for telephone service in Southeast Asia (which still has lots of 2G in rural areas), if individuals could offer hotspots / base stations with uplinks to these satellites.

Edit: change Uber-like to distributed. Side comment, lots of Uber drivers buy cars. Uber even helps finance cars.

<comment><USER44> I don't see the connection to Uber's model, unless you mean Franchise, but I doubt people already have the satellite-wifi transponder like drivers already had cars.

</comment></comment><comment><USER45> Does SpaceX have any concerns about the Kessler Syndrome [0] or any plans to mitigate it?

[0] [https://en.wikipedia.org/wiki/Kessler_syndrome](https://en.wikipedia.org/wiki/Kessler_syndrome)

<comment><USER22> Of course they do, it's a well-known problem for the satellite industry. From the technical addendum:

> _Each satellite in the SpaceX System is designed for a useful lifetime of > five to seven years. SpaceX intends to dispose of satellites through > atmospheric reentry at end of life. As suggested by the Commission,50 SpaceX > intends to comply with Section 4.6 and 4.7 of NASA Technical Standard > 8719.14A with respect to this reentry process. In particular, SpaceX > anticipates that its satellites will reenter the Earth's atmosphere within > approximately one year after completion of their mission – much sooner than > the international standard of 25 years. After the mission is complete, the > spacecraft (regardless of operational altitude) will be moved to a 1,075 km > circular orbit in its operational inclination, then gradually lower perigee > until the propellant is exhausted, achieving a perigee of at most 300 km. > After all propellant is consumed, the spacecraft will be reoriented to > maximize the vehicle's total cross-sectional area, a configuration also > stable in the direction of aerodynamic drag. Finally, the spacecraft will > begin to passivate itself by de-spinning reaction wheels and drawing > batteries down to a safe level and powering down. Over the following months, > the denser atmosphere will gradually lower the satellite's perigee until its > eventual atmospheric demise._

[http://licensing.fcc.gov/myibfs/download.do?attachment_key=1...](http://licensing.fcc.gov/myibfs/download.do?attachment_key=1158350)

<comment><USER46> > "for a useful lifetime of five to seven years"

Isn't that a relatively low lifetime for so many satellites? Doesn't that mean they'll need to make a _lot_ of launches just to keep the fleet operational?

<comment><USER47> This plan is obviously predicated on viable first stage reuse. Estimates vary (and I'm sure even SpaceX doesn't know what the eventual number would be), but it's conceivable that their per-launch cost would be in the low double digit millions. These are also relatively small satellites, so you can pack quite a few of them on a dedicated launch, as well as sending one or more uphill as secondary payloads for launches that have useful trajectories.

</comment></comment></comment></comment><comment><USER48> What is the FCCs regulatory remit when it comes to satellites? Presumably the application only affects satellites with a US footprint, and only if broadcasting on regulated frequencies?

</comment><comment><USER49> Any idea what amount of bandwidth they'll have put together?

<comment><USER50> My guess is nothing less than 1 Giga bit per second.

</comment></comment><comment><USER51> Am on satellite internet. When it works, it's kinda okay. 25mbps down are possible and sometimes achieved. On a Friday evening though, the shared medium hits hard. Last week an 'aptitude update' took an entire business day.

So, yeah, I'm gonna assume people who're by every definition smarter than me aren't going to try this kind of 'meh' technology for their own Google Fiber like plans. Ground based RF networks make a lot more sense imho.

<comment><USER52> The SpaceX plan puts the satellites in low earth orbit, much closer than the geosynchronous satellites you can connect to now. That lowers latency into the 25-30ms range. There will also be many more of them, and presumably they are being built with the understanding they will need to support the bandwidth requirements of the Netflix-era of internet usage.

<comment><USER51> Thanks for clearing that up. Latency is another of these things currently, pings below 800ms are a rarity for me. Very noticeable and pretty annoying.

I'm not presuming I know better than a bunch of SpaceX engineers.

</comment></comment></comment><comment><USER53> Would anyone be surprised if this is (in part) responsible for the shift in focus for Google Fiber? Why would Google invest Billions for a small portion of coverage while Elon is busy building an ISP for the whole world? Elon and Larry are friends, and Elon has talked about using LEO satellites for internet access before.

<comment><USER16> Musk says that 90% of connectivity in cities will be fiber, only 10% satellite. Fiber works great in high density areas.

</comment></comment><comment><USER54> So, would this be consumer-facing internet, or would SpaceX be subletting capacity on their network for various providers to offer "space internet"?

This has been mentioned before in the context of municipalities building/maintaining fiber infrastructure and subletting to ISPs.

</comment><comment><USER55> Musk's 4.4k strong array of communications satellites in LEO becomes a weapon (think Lex Luthor) if you can point the full transmit power of a bunch of them at a single point (think MIMO).

How much microwave energy do you need to put on a watermelon to make it pop?

</comment><comment><USER56> Where is the satellite quantity for this application? I cant find it.

</comment><comment><USER57> This got me to think if it is possible to create a super giant Wireless Network, With Satellites, and Microwave tower on ground as well as Whitespace Super WiFi / LTE mesh network.

</comment><comment><USER58> SpaceX might target phone business also besides internet

</comment><comment><USER59> protip: buy rural real estate in good locations before this goes live.

<comment><USER60> That real estate already has coverage by Verizon, AT&T and probably a couple of WISPs.

Source: I live in such a place.

</comment></comment><comment> hash-set The problem with satellite Internet is the uplink and downlink speeds.

</comment><comment><USER61> Can't wait for global internet!!!!

<comment><USER62> ... as opposed to local internet that we have now?

</comment></comment><comment><USER63> Maybe they should start with a reliable launch system.

<comment><USER64> If they'll be launching thousands of satellites, I expect the marginal cost of replacing one launch to be lower than what most satellite makers could manage. And then, reliability has a cost of its own.

Have you calculated the optimum?</comment></comment>


Doppio: A Java Virtual Machine, Compiler and Disassembler in JavaScript - <AUTHOR> http://badassjs.com/post/32812527381/doppio-a-java-virtual-machine-compiler-and

<comment><USER2> I liked the part where it suggested a JS JVM was useful since JS could be compiled for the JVM.

<comment><USER3> It is only a matter of time until somebody gets a webbrowser running inside another with javascript as a joke.

What will be interesting is to see how long we have to wait afterwards until someone decides that it might be a useful thing to do in production.

<comment><USER4> Get a performant IE6 running inside Chrome, and I'd pay good money for that.

<comment><USER5> While it's not running INSIDE the browser per-say there is <http://www.browserling.com/> that'll let you use another browser inside another browser (and use browserling inside that other browser if you need to go deeper).

</comment></comment></comment></comment><comment><USER6> I don't see anything like a Java compiler in the Doppio source.

edit: I get it. In the demo they are using a Java compiler from Sun that is itself a Java program running in the Doppio JVM. Neat.

<comment><USER7> Check out the demo. It definitely contains javac <http://int3.github.com/doppio/>

</comment></comment><comment><USER8> What a wonderful world, on one end you have people moving client side code to the server (node.js, Dart, Meteor) on the other hand you have people trying to move it to the client (GWT, Doppio)

<comment><USER9> I'm thinking the client people have the edge. If you're running a web site, why pay for a big machine when you can let each client use their own.

<comment><USER10> Because the alternative is writing javascript _shudder_.

</comment></comment></comment><comment><USER11> Does this mean it might be possible to run java in the browser with out it grinding your computer to a halt?

<comment><USER12> "I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."

<comment><USER8> \- Charles Babbage, Passages from the Life of a Philosopher

</comment></comment></comment><comment><USER9> This is so cool how can you not like the concept. I even tried "globalthermonuclearwar" but it just wanted to play Tic Tac Toe. :-)

</comment><comment><USER13> I guess the letter campaign _was_ the slower path to getting Java on iOS.</comment>


Underscores are stupid? Get a Japanese keyboard - <AUTHOR> http://recursive-design.com/blog/2012/05/16/underscores-are-stupid-get-a-japanese-keyboard/

<comment><USER2> If youre going to use a Japanese keyboard, why not go all the way and use Kanji in your code also? Instead of (courtesy of Wikipedia)...

      def create_set_and_get(initial_value=0) # Note the default value of 0
        closure_value = initial_value
        return Proc.new {|x| closure_value = x}, Proc.new { closure_value }
      end

...we could write (excuse my Chinese)...

      做 创_去_和_来(先_值=0)
        闭_值 = 先_值
        回 程.新 {|x| 闭_值 = x}, 程.新 { 闭_值 }
      终

Oh look: all the "words" are a single character each! Why not get rid of all those silly grammar symbols in between them (just like in Chinese and Japanese writing)...

      做创去和来(先值=0)
        闭值=先值
        回程新{|x|闭值=x},程新{闭值}
      终

Now there's tons of whitespace on the righthand side, so why not make it all a single line, with spaces instead of newlines...

      做创去和来(先值=0)  闭值=先值  回程新{|x|闭值=x},程新{闭值}  终

A lot terser! Makes all the talk about semicolons and underscores sound a little silly to some people, doesn't it?

<comment><USER3> Already exists: <http://www.chinesepython.org/>. (Scroll down for a short translated code snippet.)

English info: <http://www.chinesepython.org/english/english.html>.

<comment><USER2> Chinese Python is a Chinese version of Python, but doesnt utilize the inherent tersity of CJK characters. For example it translates "import sys" as "載入 系統". The example I gave would eliminate spaces, semicolons, underscores between words by using one and only one CJK character per English word, so in the example "import sys" would become, say, "入統".

</comment></comment></comment><comment><USER4> Or just reprogram your keyboard (xmodmap) to make any button do anything you want.

<comment><AUTHOR> I'm curious - do you physically take the keys off and swap them when you do major mapping changes like this to make using it easier? Or just use touch typing/muscle memory and ignore what's written on the keys?

<comment><USER5> I use Colemak and occasionally one-handed Dvorak (when I'm having RSI issues in one hand or the other), but my keys are laid out QWERTY. Move the keys, and touch typing becomes much more difficult because you lose your "homing bumps" at the f and j positions. Also, learning a new layout is easiest with a diagram floating on the screen, not by trying to look through your fingers at the keyboard.

<comment><USER6> I am another Colemak user. I use a Truly Ergonomic keyboard on my desktop, and having to look at the keyboard because of the odd placement of enter, tab, quotes, etc. absolutely kills my throughput compared to my laptop.

<comment><USER5> Interesting. I had not seen that kind of keyboard before. Seems like having "b" on the left hand would also be distracting, but maybe that's how other people type.

</comment></comment></comment></comment></comment><comment><USER7> I don't like his statistics so much:

Ruby - Saving 34% (6,613 keystrokes) PHP - Saving 52% (105,250 keystrokes)

Those percentages are not the most helpful. They represent a relationship between the positive delta and the negative delta, rather than a savings for the project (which I feel is the first implication in a percentage saved).

<comment><AUTHOR> Yeah I don't really like them either, and I wrote them. I thought twice about including the percentages, and I did try to highlight that fact:

        Note that the percentages below are only relative to
        these 2 sets of 3 characters, and don't represent the
        total percentage of keystrokes saved.

The percentages are only really there to provide an easier way to compare the (relatively small) Ruby sample, and the (relatively large) PHP sample. Simply saying that one project saved 6000 strokes vs. the other with 100,000 strokes seemed a bit meaningless given the relative sizes of the projects, and doing a full-on frequency analysis of modifier key usage seemed like more trouble that it was worth... perhaps in hindsight I should have done it anyway.

zokier provided an interesting link to a previous discussion below (<http://news.ycombinator.com/item?id=2973776>, specifically <http://news.ycombinator.com/item?id=2977303>) which does a much more thorough analysis of Finnish/Swedish vs. US layouts - I'd like to run his code using the Japanese layout when I get a chance.

</comment></comment><comment><USER8> see this previous discussion for keyboard/programming stats: <http://news.ycombinator.com/item?id=2973776>

Especially the methodology I used then to compare Finnish/Swedish and US layouts seems similar to his.

</comment><comment><USER9> Actually Yukihiro "Matz" Matsumoto, designer of Ruby is Japanese. It's pretty ironic, isn't it?

<comment><USER10> It's not ironic at all. One would expect that authors using a particular keyboard layout would tend to favor characters that are easy to type. In that way, it's the opposite of irony. The outcome is exactly what we'd expect.

<comment><USER11> See also the keyboard used to develop the original vi:

<https://en.wikipedia.org/wiki/File:KB_Terminal_ADM3A.svg>

<comment><USER9> That explains also the tilde=home :)

A recent article I've read somewhere explains it very well!

</comment></comment></comment></comment><comment><USER12> I have the number keys mapped to their symbols (i.e. pressing 1 gives !, pressing Shift+1 gives 1 etc.) using a custom XKB map. Feels so much more efficient, although I don't have any hard stats.

<comment><USER13> Sounds almost like a FR layout.

<comment><USER14> FR layout is great for this (and writing lisp is easier when the parenthesis are easily typed) but the []{} characters are horrible to type!

</comment></comment></comment><comment><USER15> Interesting, according to the stats, if you have = sign done with out the shift it would have saved another 20% of the keystrokes.

</comment><comment><USER16> This is my current .Xmodmap file

keycode 48 = quotedbl apostrophe quotedbl apostrophe

keycode 66 = Tab Caps_Lock NoSymbol Caps_Lock

keycode 20 = underscore minus underscore minus

</comment><comment><USER17> Use dashes for separators like in Lisp.

<comment><USER18> Just it doesn't work in most languages, as the - sign is usually the minus operator. In Java and Python 3 for example you can use Unicode in identifiers, maybe you can use mdash there (unless they forbid punctuation characters).

<comment><USER19> For Emacs users, there is the wonderful package Smart Dash Mode: <http://malsyned.net/smart-dash.html>

It takes a pretty good stab at whether you want a dash or an underscore, based on context.

It's a whole lot easier on the wrists.

(The only problem is you start to wish you had it everywhere, not just in Emacs.)

<comment><USER18> Wow, as an Vim user, I think this is seriously cool. Now it'd need to support camelCode as well, so when I write camel-code in JavaScript, I get the right thing. That'd be awesome.

</comment></comment></comment></comment><comment><USER20> > _The Control key is in a far more comfortable position below the Tab key, > where caps lock would be on a traditional keyboard_

Well, there's tradition and there's tradition... Traditional Unix systems usually had ctrl there, and the old Symbolics/"MIT space cadet" keyboard actually had what I believe is the equivalent of backspace ("rub out") in that location. The modern location of caps lock seems to be a PCism.

Edit: Forgot to mention, notably even the Apple II and III had ctrl there, I don't think Apple switched to the "modern" layout until the Mac:

<http://www.flickr.com/photos/suburbia_org_uk/4820757193/>

[http://www.old- computers.com/museum/photos/Apple_III_Keyboar...](http://www.old- computers.com/museum/photos/Apple_III_Keyboard_s1.jpg)

</comment><comment><USER21> Don't get a Japanese keyboard... just get a better layout!

<http://bitbucket.org/cheater/us_split/overview>

It's just like qwerty, but better, because it inflicts less pain.

I didn't even know, but Apple started using it with iPads! [http://www.gottabemobile.com/2012/02/03/ipads-split- keyboard...](http://www.gottabemobile.com/2012/02/03/ipads-split-keyboard-has- hidden-buttons/)</comment>


Show HN: Responsive Watsi, my (humble) attempt at a mobile first rebuild - <AUTHOR> http://watsi.substructu.re/

<comment><USER2> Wow, this is pretty slick. The sign-up/login forms are awesome, works great on my HTC One. You should get Watsi to adopt this.</comment>