Why we need Python in the Browser - <AUTHOR> http://archlinux.me/dusty/2012/03/13/why-we-need-python-in-the-browser/

<comment><USER2> For some time now I've been wondering why the primary scripting format consumed by browsers is JavaScript (a language intended for humans) and not some sort of standard bytecode that would be executed in a browser's virtual machine.

What we need is the JSVM! In other words, something like the JVM but adapted for the browser. That probably means it would have a more dynamic typing focus since it makes sense for JavaScript to be the primary language targeted by the system (though there's no reason this couldn't evolve over time just like the JVM has).

All of a sudden you don't need to build in Python support, or Ruby support or LISP support or whatever else you fancy into the browser. The browser no longer cares what you compiled the script from, it just gets the standard bytecode and everyone's happy.

<comment><USER3> I'm sure we could have some sort of JSVM, we could even have some sort of bytecode for the HTML, but it would destroy part of what makes the web so great: it's openness. The ability to view the source of any webpage, and even make chances and enhance it locally.

There is also the fact that certain optimization can only be done by the browser if it has the actual source code rather than the compiled bytecode.

<comment><USER4> In both .net and java there exists excellent decompilation tools where you can easily decompile the bytecode back into java or C# or anyother language. It's not necessarily going to give you the original source code (especially missing comments) but it works incredibly well nonetheless. You could easily debug in it and edit it dynamically in browser tools as well. This is pretty much a solved problem. I wouldn't consider this a limitation worth talking about again.

</comment></comment></comment><comment><USER5> Python in the browser would ultimately lead to all the same problems as Javascript in the browser. In a few years, we'll be hearing: Clojure in the browser, Scala in the browser, Haskell in the browser.

I've been programming in Python full-time for the past 6 months, and it's main deficiency are the same as JavaScript's: no (optional) static typing - I'm really sick of writing checks whether my function got values of the correct types, and unittests testing if a class's/function's signature changed in an unexpected way.

What we need, in my opinion, is a dynamic language with powerful optional static typing (unlike Dart, where types are just comments), with sane object- orientedness, and support for immutable values. The core has to be really simple, but the language has to be powerful enough so that libraries can provide the missing functionality (math - rationals, matrices, ...; concurrency - channels, isolates, ...; GUI, IO (with formats, ...)).

<comment><USER6> > I'm really sick of writing checks whether my function got values of the > correct types, and unittests testing if a class's/function's signature > changed in an unexpected way.

Sorry to be blunt, but if every one of your Python functions and methods begins with isinstance() checks for every parameter, you just missed the point of the entire language. Python was built on a foundation of ducktyping, with the mantra that "it's easier to ask forgiveness than permission" (EAFP).

Just use your parameters in whatever way you expect them to be, and either catch the rare Exception there or allow them to bubble up. Here's some suggested reading on the subject:

<http://www.canonical.org/~kragen/isinstance/>

[http://stackoverflow.com/questions/6092992/why-is-it- easier-...](http://stackoverflow.com/questions/6092992/why-is-it-easier-to- ask-forgiveness-than-permission-in-python-but-not-in-java)

<comment><USER7> The Python approach means that when an exception is thrown, it's unclear whose fault it is. Is it a bug in the called function, or in the caller for passing a bad parameter? The larger the program the more of a problem this becomes.

It also means that when you change a function signature, the compiler can't tell you all the call sites you need to fix. You have to find them through testing. This is a large barrier to aggressive refactoring.

Even the Python C API provides facilities for checking that arguments have a specific type and raising TypeError if not, so it can't be _that_ completely counter to the intention of the language: <http://docs.python.org/c-api/arg.html> (see O!)

<comment><USER6> > It also means that when you change a function signature, the compiler can't > tell you all the call sites you need to fix. You have to find them through > testing. This is a large barrier to aggressive refactoring.

This is a problem inherit to every dynamic language. You sacrifice brain-dead refactoring for improvements and tradeoffs elsewhere. Not to mention that modern IDEs and tools like Rope [1] can definitely help.

[1] <http://rope.sourceforge.net/>

</comment></comment></comment></comment><comment><USER8> We don't need Python in the browser. We need x in the browser, where x is every language and runtime.

Javascript isn't some sort of global optimum that's perfect for every possible application. The subset of Javascript that's supported by a wide range of current browsers certainly isn't. And the result is that people aiming for the web environment are limited by what Javascript can do - even though there are languages that are arguably more expressive than it.

The problem is right now, only browser makers get to decide what languages run in the browser. Browser makers have their own concerns, and supporting new languages hasn't been historically among them.

What would be nice to aim for is a model where browsers can support multiple language runtimes. Instead of the browser makers being the ones to support a language, that language's advocates would be responsible for the port - and I suspect the competition and cooperation would make them stronger.

Ideally, the language runtimes would be installed transparently. That's the big potential of a project like Native Client - if it lives up to its billing, it makes downloading the latest version of a language runtime to a browser safe, while giving near-native speed and abstract machine.

This would let Python, Ruby, Haskell, Scala, Closure, Java, non-legacy Javascript, and more exist in the same browser, giving the web platform the same diversity of languages as is available on the desktop.

(Mobile platforms seem to also have this problem. The recent move from general-purpose to language-specific platforms is sub-optimal.)

<comment><USER9> So I'm a little ignorant, but I think I'll benefit from answers more informed people give:

My question is, why aren't people out there (who are more experienced at programming/developing than I am) developing support for X language(s) in open source web browsers like Firefox. I'm sure I don't fully understand how this works yet, but can't anyone who wants to be developing this capability independently instead of waiting for browser makers to do it?

<comment><USER10> Because unless the major browser makers ship it in their mainline product, it's worthless.

<comment><USER9> That's a good point; does that imply that it can't be incorporated after the fact? That if a number of dedicated users applied it to the product themselves, the browser makers wouldn't come around to incorporating it?

</comment></comment></comment></comment><comment><USER11> We don't need Python in the Browser.

We need a _VM SPEC_ for the browser, so we can have whatever language we damned well please in the browser.

Why Google decided to go with Dart rather than publish a VM spec is beyond me.

<comment><USER12> Brendan Eich has talked about why bytecode + VM is a bad idea for a browser: <http://www.aminutewithbrendan.com/pages/20101122>

If you really want a VM, it already exists. It's called JavaScript, assembly language of the web.

I would hate for sites to start saying, "Requires the PyJSVM v0.8 or better to function, download it now!"

EDIT: in hindsight, this post was careless. I was wrong about the "PyJSVM" point, and I posted the minutewithbrendan link to provide commentary, not "Eich said it, so it must be true."

What I should have said:

JavaScript is so widely used today, that anything that comes along purporting to be better (such as bytecode, Dart, whatever) must be _so much_ better as to provide a clear reason for developers and users. If it's just "better", then JS will remain dominant because it's good enough. Therefore, it makes sense to target JS from other langs. It's definitely not getting slower, and we're on the cusp of some great APIs!

<comment><USER11> Surprise, surprise. Mr. Javascript thinks a VM for the browser would be a bad idea. Let's address his points:

Viewing source is a non-sequitor: the source code could be streamed down with the bytecode.

Standardizing a bytecode is no harder than standardizing a language and DOM, in fact, should be easier.

Versioning bytecode is no harder than versioning languages.

Bytecode does _not_ imply an implementation any more than a language does, although it can imply semantics.

And then, at the end, he basically advocates a JS-specific bytecode. Hah. Oh, but they aren't working on it in committee. That says about all you need to know.

There is _nothing_ about a bytecode spec that harder than a language and language spec.

Basically, he doesn't want any competition for Javascript. That was an incredibly weak podcast and he should have been shredded for it. People cut this guy way too much slack.

<comment><USER13> The semantics are the hard part. Just making a bytecode for some language isn't hard. The problem is coming up with some semantics that allow all languages to be implemented efficiently on top of it. Turns out that's extremely difficult. The JVM and .NET certainly didn't achieve that; the Java- fied and .NET-ified versions of languages are typically slower and don't integrate well with their host languages.

<comment><USER11> Slower and integrate less well than _javascript_?

C'mon.

</comment></comment></comment></comment></comment><comment><USER14> There's a Firefox extension that integrates Python: <http://code.google.com/p/pythonext/>

With that installed, you can write extensions in Python and even include Python code in your HTML files if you want to: <https://developer.mozilla.org/en/PyDOM>

</comment><comment><USER15> Source mapping (part of ES6) gives proper error messages for other languages compiled into JS, I believe this would include Pyjamas etc.

But do Python programmers who haven't programmed for the browser before realize how important async is?

Tornado and Twisted users are used to passing around functions, but they're very small subsets of the Python community, who, if asked to fetch something and do something with it, would generally wait until it's fetched.

<comment><USER16> "But do Python programmers who haven't programmed for the browser before realize how important async is?"

Not a very useful question. Anyone who has ever programmed on a GUI of any sort will know how important that is, anybody else will rapidly learn. The vague-but-pervasive idea that web developers invented async about two years ago and thus "async experience" can be presumed to have not penetrated out to those other less enlightened communities who still haven't discovered its Mighty Powers is... not exactly historically accurate, let us say. GUIs have always had to deal with async issues, going all the way back to the dawn of GUIs. It's not a new idea. It's probably older than you are. (It is older than I am.)

<comment><USER15> > Anyone who has ever programmed on a GUI of any sort

Most Python folks are backend folks. As mentioned, I frequently see Python folk (including myself a few years ago) confusing async code for JS being a complex language.

> anybody else will rapidly learn.

Why?

Delphi and the other desktop app RAD tools took care of callback integration - the entry point for the code was generally not shown, it was just visually attached to the button.

</comment></comment></comment><comment><USER17> Can the rpython translator support js as a target? Could you compile pypy to js that way, to provide a python environment in existing browsers?

To be honest though, JS is a perfectly OK language for web browsers, and if you really want to reuse python code it probably makes more sense to translate it before it gets to the browser, using automated tools or otherwise. Supporting everybody's favourite language natively in every browser would be horrible.

<comment><USER18> Yes, you can compile PyPy to JavaScript. There even used to be a JS backend as part of the RPython translation toolchain, but it got dumped since no-one was interested in working on it [1]. There've been more successful attempts like RPythonic [2] in translating from RPython to JavaScript by going via Emscripten [3], i.e.

      RPython -> LLVM -> Emscripten -> JavaScript

And, then, there have been efforts like my own which have focused on building a WebKit bridge [4] to PyPy.

[1] [http://www.mail-archive.com/pypy- firstname.lastname@example.com/msg03946....](http://www.mail-archive.com/pypy- firstname.lastname@example.com/msg03946.html)

[2] [http://pyppet.blogspot.com/2011/04/rpython-to- javascript.htm...](http://pyppet.blogspot.com/2011/04/rpython-to- javascript.html)

[3] <https://github.com/kripken/emscripten>

[4] <https://github.com/tav/naaga/tree/master/webkit_bridge>

</comment></comment><comment><USER19> I like Python, but JavaScript is sort of good at what it does: evented programming. In javascript it's really easy write a closure. I have not worked with asynchronous frameworks like twisted, but I'd image it's a little more awkward. Am I wrong?

<comment><USER4> Python also has lambda syntax, therefore closures are quite easy to do. Example:

items.select {|e| e.isFoo}

It's arguably more terse and superior to javascript in that particular feature as well.

<comment> steve-howard That's Ruby, if I'm not mistaken. In Python you'd do

filter(lambda e: e.isFoo, items)

<comment><USER20> It's more Pythonic to write

        [e for e in items if e.isFoo]

</comment></comment></comment></comment><comment><USER21> There's <http://skulpt.org>. Not sure how complete it is, though.

</comment><comment><USER22> The language in the browser is not the problem to me. If I could use python it wouldn't change the fact that you still need to deal with the DOM, CSS and browser differences.

</comment><comment><USER23> It seems that the primary method of disagreement used by only-JavaScript-in- the-browser advocates is as follows:

1\. Talk about the "open web"; 2\. Tell you to compile to JavaScript.

I really don't understand why this is considered a reasonable response to people who just want to build things with the language they prefer, without being treated as second-class citizens.

<comment><USER24> Maybe because people are afraid to end up with a fragmented web all over again.

Preference of language shouldn't really be a factor. The real question is that any change must be properly standardized and implemented by all browser vendors else you end up with "This webpage is written in python and is only supported by Chrome 39+ and Firefox 43+".

Does using your prefered language really outvalue that?

I think the bytecode argument is a more feasable one, as long as it could somehow be compiled to javascript and support older browser versions effortlessly.

<comment> steve-howard Version fragmentation is nearly a non-issue if browser updates are automatic by default and non-intrusive. Firefox is slowly catching up, and Chrome is very good at this. IE would obviously need to substantially change its release cycle if it lost enough market share, though I imagine that won't happen for quite some time.

</comment></comment></comment><comment><USER25> If google is serious with Dart, I'd hope they would release some library/tool that bridged python to chrome and supported the dart dom libraries.

I would think that would help make it popular which would help compel other browsers to support such technology.

</comment><comment><USER26> This is ridiculous. I love python, but it has no language support for an event loop, and if you put it in a browser, everything would have to be done in an event loop. It'd be ugly.

Javascript excels at evented programming. Use the right tool for the job.

<comment><USER27> I was just going to comment that at least Python has _yield_ (generators) which helps to avoid callback spaghetti¹, but Javascript seems to have got it too in 1.7².

[1] <http://www.tornadoweb.org/documentation/gen.html>

[2] [https://developer.mozilla.org/en/New_in_JavaScript_1.7#Gener...](https://developer.mozilla.org/en/New_in_JavaScript_1.7#Generators)

</comment></comment><comment><USER28> I think about this, a time ago and create a drumbeat project:

[http://webcache.googleusercontent.com/search?q=cache:Q-BEBq4...](http://webcache.googleusercontent.com/search?q=cache:Q-BEBq4_Bb4J:https://batucada.mozillalabs.com/en- US/projects/client-side- python/+mozilla+drumbeat+python&cd=3&hl=es&ct=clnk&gl=ar&client=firefox-a)

but, sadly, nobody follows this idea...

</comment><comment><USER29> If your python eventually compiles down to JavaScript it's worth doing. That way you can get the developers to hop on because it's easier to program. Browsers could then implement functions to directly process the python source and not the JS which would have speed benefits.

But I really feel there is no need for python in the browser. JS is good enough. If you want an easier way to write it use CoffeeScript.

</comment><comment><USER30> With a tool like Enyo you can write all the code in a sort of JavaScript, and it is converted to HTML + JavaScript. Google also has a tool to convert Java to JavaScript. Why not a similar tool for Python (or other language)? No practical need to have a lot of languages in the browser itself IMHO even if it would be nice to program web pages with his favorite language.

</comment><comment><USER31> An ideal situation would be installable interpreters that still allowed you to use a system-wide rendering engine. I actually think Windows had something similar with ActiveScripting.

I assume something like QT allows you to script the webview with whatever language you have bindings for? Unfortunately QT+bindings is a pretty long way from 'zero install'

<comment><USER32> Sounds more like plugin hell to me. I think it's becoming increasingly harder to convince users to install browser plugins (for good reasons), and it's not even possible on most mobile browsers.

<comment><USER31> As with the original article I'm talking about desktop apps which would already require an installer. It's just about reducing the pain of that install.

</comment></comment></comment><comment><USER33> We don't need another language in the browser. We need a VM in the browser. As in byte codes. That way ANY language can be used. JS gets compiled down to a VM and then that is run in most implementations of JS. Why can't we just target that VM directly and use our own tools and compilers instead of being forced to compile it to javascript?

</comment><comment><USER34> Sorry if this is a bit off-topic, but its mentioned that there is work being done on an event loop in python, Can anyone give me some sources on this?

Are they talking about async frameworks such as twisted/tornado? or is there more core work that I don't know about?

</comment><comment><USER35> JavaScript is the bytecode that you can compile your python or whatever code too. Just make the standards committee implement some nice features that are useful for bytecodes and you are all set.

</comment><comment><USER36> I remember Internet Explorer used to support VBScript. Is that still supported in IE? It's been so long since I've used IE!

</comment><comment><USER37> What about using Native Client (nacl) to compile an implementation of python (without the batteries)?

<comment><USER38> It would just run on Chrome, and it would just run on x86 and x86_64 machines.

<comment><USER37> That's true but for the first part, it's mostly due to a lack of interest by other vendors. For the second part, isn't the nacl team working on a protable version of nacl?

It's at least one way to get python in the browser today.

<comment><USER38> > That's true but for the first part, it's mostly due to a lack of interest by > other vendors.

The lack of interest by other vendors is because NaCl isn't an appropriate technology for the open web, not just random lack of interest. (First and foremost, because it is CPU-specific, while the web is supposed to run everywhere.)

> For the second part, isn't the nacl team working on a protable version of > nacl?

A research project called PNaCl, yes. But it is a different technology than NaCl despite the similar name. One is based on gcc, the other llvm, one ships CPU-specific machine code, the other bitcode, etc. So it is at this point too early to tell if PNaCl will achieve good portability+speed, NaCl's speed doesn't mean PNaCl will be fast too. Adopting NaCl because of the promise of PNaCl doesn't make much sense.

</comment></comment></comment></comment><comment><USER39> how would one go about minimizing python code ?

</comment><comment><USER40> Would creating a Python interpreter that runs on Chrome's Native Client be enough? In NaCl you have an environment that lets you take C/C++ code and deploy programs written in that code to the browser. The Python interpreter (and the ones for several widespread languages) is written in C. Why not put the two together?

I've seen a proof-of-concept of this on webOS, which is essentially a browser since JavaScript programs can run natively. In webOS 1.4.5 and above, there is the PDK, which is a library that lets you make C/C++ programs compatible with webOS and run them on-device. Part of the PDK is the plugin interface, which allows JavaScript code to call C-compiled functions, which the C-compiled program exposes to JavaScript with special PDK functions.

Since the Python interpreter is in C, it can be adapted with the PDK to allow Python programs to run natively. Here is a post demonstrating it:

<http://www.ezequielaceto.com.ar/techblog/?p=359>

In Chrome, NaCl is analogous to the PDK, so why not have Python as an NaCl plugin and use it to run Python in the browser? You have to imagine that if NaCl gets widespread use, similar and compatible technology will pop up in competing browsers, so sooner or later your Python will be cross-browser.

I'd like to see someone try Python in NaCl.

</comment><comment><USER41> Just don't be so afraid of JavaScript. The future can be yours, today!

</comment><comment><USER42> > the most restrictive operating systems only permit us to code in Objective C

uninformed gibberish.

</comment><comment><USER4> I think he misspelled 'Lua'.

</comment><comment><USER43> Many people would be very surprised to see what MS Pyton is capable of (-:

<comment><USER15> Sure, Silverlight can run Python, but Microsoft has stepped away from plugins in IE10, and SL/ML run poorly or not at all in most other desktops and tablets.

</comment></comment><comment><USER44> As a user of Real Programming Languages (Common Lisp, Java, C) I'm indifferent to scripting languges, but if I had to choose ONE to see in the browser, it would be Lua.

It has sane, symmetric minimal design that gets closures. It also _looks_ like a programming language. I can type Lua into the Firebug repl and not worry about indentation.

<comment><USER45> I like how you capitalized the Real Programming Languages, as if it is some official definition.

The notion that python isn't real, is just silly.

<comment><USER46> Of course Python is a Real Language, it exists, etc, yes. But it's not a Real _Programming_ Language.

You see, everyone using python, ruby, perl, php, and other similar platforms - we're not _programming_ , we're just _scripting_. The difference is _so_ extreme, that, well... if you don't understand, _I'm_ not going to bother explaining.

<comment><USER47> Sorry, Poe's law kicked-in and I mistakingly downvoted you. I'm very sorry:|

<comment><USER46> Sorry too - I should have added a sarcasm tag (or, more appropriately, a sarchasm tag).

I've been doing PHP for 16 years (among other things), and have grown tired of the "scripting vs whatever" debates/comments/remarks/jibes/etc over the years. That didn't come across enough in my original post - sorry. :/</comment></comment></comment></comment></comment>


Trump reportedly preparing new ways to block China's interest in American tech - <AUTHOR> https://www.technologyreview.com/the-download/610594/trump-is-reportedly-readying-new-ways-to-block-chinas-interest-in-american-tech/ <comment><USER2> Can someone weigh in here? Seems like if china wanted to slap a tax on every cpu chip it would hurt a bunch of our businesses that rely on fabs in China (intel, apple, etc.)?

<comment><USER3> I think the intent is to make it so that these companies find other places to manufacture. Yes I know this can't happen overnight but it would get the ball rolling.

</comment></comment><comment><USER4> China has brought this upon itself. The Chinese do not play by the rules, so we don't really have a choice but to punish them.

People rolling over on tariffs because they're afraid of a trade war are entirely missing the bigger picture. In order to enforce rules, you need carrots and sticks. A WTO member country continually acting against WTO mandates cannot be tolerated. If we get a trade war, so be it. The system must be respected. Just because China is massive and imposing doesn't mean it gets to flout the rules everyone else plays by.</comment>


Add HN-style "about" field to user profiles [on reddit] and remove the 10:1 rule - <AUTHOR> http://www.reddit.com/r/ideasfortheadmins/comments/xmsan/add_hnstyle_about_field_to_user_profiles_and/

<comment><AUTHOR> I can post this again if anyone here wants me to.</comment>


How to Self-Publish a Novel in 2017 - <AUTHOR> http://www.zhubert.com/blog/2017/02/25/how-to-self-publish-a-novel-in-2017/ <comment><USER5> The author mentions getting a copy editor, but I would also advise hiring a development editor and/or getting feedback from beta readers. Quoting Stephen King (1):

 _Show your piece to a number of people—ten, let us say. Listen carefully to what they tell you. Smile and nod a lot. Then review what was said very carefully. If your critics are all telling you the same thing about some facet of your story—a plot twist that doesn't work, a character who rings false, stilted narrative, or half a dozen other possibles—change that facet.

It doesn't matter if you really liked that twist of that character; if a lot of people are telling you something is wrong with your piece, it is. If seven or eight of them are hitting on that same thing, I'd still suggest changing it. But if everyone—or even most everyone—is criticizing something different, you can safely disregard what all of them say._

In the same piece, King advises "if it's bad, kill it." ("When it comes to people, mercy killing is against the law. When it comes to fiction, it is the law.")

1\. [http://www.jerryjenkins.com/guest-blog-from-stephen- king/](http://www.jerryjenkins.com/guest-blog-from-stephen-king/)

<comment><USER6> With this approach, your novel becomes an average of what the masses like to read, rather than what you wanted to say. Engineered for the masses, so to speak. Which, of course, could explain why King is so successful.

<comment><USER7> IMO, this is an artistic fallacy that needs to die. Responding and adjusting to feedback doesn't undermine the integrity of a work or weaken its authenticity. Cycles of feedback and iteration are common to all work in every medium.

Obviously, you can take responding to feedback too far, but the idea of the artist-in-a-tower-birthing-a-fully-grown-work-whole is an inaccurate idea at best, and a damaging assumption at worst.

Like King says, ignore the outliers. But if _no one_ understands a plot twist, or _everyone_ dislikes something you really like, maybe take another swing at it. Iterate a little more. Creative output is the art of refinement as much as anything else.

(Good editors can also often help substitute for peer feedback)

<comment><USER8> Depends on what you account for as feedback. If we are narrowly talking about plot twists when writing an entertaining thriller, than yeah, by all means, make sure the audience gets it. Also, there is much to be said about a sharp, crisply edited book that doesn't meander or ramble pointlessly.

On the other hand, in general I almost exclusively read books and blogs because I'm interested in what the author has to say, rather than what I'd like to hear. Cal Newport discusses the idea of craftsmanship in a way I really like. When you're building a table or a bass guitar, there are better and worse woods for the application at hand that have little to do with how the craftsman feels about them. They are, to an extent, impersonal and external to him. A song or a poem, by contrast, have everything to do with the author and little in the way of objectivity about them.

Of course, all of this is a case of finding an optimum balance.

<comment><USER9> >On the other hand, in general I almost exclusively read books and blogs because I'm interested in what the author has to say, rather than what I'd like to hear.

I assume you read mostly nonfiction, then?

<comment><USER8> I'd say the author's voice shines through even clearer in fiction though. Authors certainly can exercise flair when writing history or something, but even more when they invent a fictional world and develop and atmosphere and characters within it.

When a fiction writer comes up with a plot, or even a turn of phrase that blows my mind, that's one of the best things about reading. Like "how did he think up all this?!"

<comment><USER7> > When a fiction writer comes up with a plot, or even a turn of phrase that > blows my mind, that's one of the best things about reading. Like "how did he > think up all this?!"

Typically the answer is: hard work and iteration, and many times not by himself. I think you're partially falling for the fallacy of the singular author completing a perfect work with unflinching vision. IMO this is incredibly rare.

Take, for example, Harper Lee and To Kill a Mockingbird. It's an amazing book. It's a novel that might at once feel effortless and also an unaltered vision of the author. But it's actually the result of years of editing ([https://en.wikipedia.org/wiki/Harper_Lee](https://en.wikipedia.org/wiki/Harper_Lee)):

> Ms. Hohoff was impressed. "[T]he spark of the true writer flashed in every > line", she would later recount in a corporate history of Lippincott.[16] But > as Ms. Hohoff saw it, the manuscript was by no means fit for publication. It > was, as she described it, "more a series of anecdotes than a fully conceived > novel".[16] During the next couple of years, she led Lee from one draft to > the next until the book finally achieved its finished form and was retitled > To Kill a Mockingbird.

Everything from the narrative structure to the title to characterization is refined, over and over again.

The goal with feedback and subsequent editing isn't the dilution of an author's vision, it's the opposite. It concentrates and clarifies the vision with each successive pass.

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> I just finished my first book, and I went a little bit different of a route than the author here.

I know this isn't very DIY, but it's probably worth knowing it's an option: I simply hired someone who works at a book publishing company to edit and typeset the book for me on the side. The person I hired does it for a living and is incredibly good at it. Let's just say it's not the most lucrative career choice, so I had the entire book (198 pages) edited and typeset by a professional for about $400.

This also had the added benefit of letting me write and get feedback from early readers using google docs, which was incredibly important.

I ended up paying her a pretty big bonus on top of that (as we pre-sold $110,000 worth and since publishing a couple days ago have brought in another $2,000 - all straight profit since we're ebook only so far and not on Amazon) but for my time and sanity it was very, very worth it.

<comment><USER11> > we're ebook only so far

Any insight on "piracy" rates? I recently published something print-only, and was considering offering a PDF (layout matters). It's a completely different "vertical," so the rest of your experience isn't too relevant.

<comment><USER12> A good take on this from a writer friend of mine: the people that "pirate" the book were likely never going to pay for it. But they could very well recommend it to people who might prefer buying it. The more it gets "pirated" the better as that means it's likely to be a decent book. I kind of agree with this and my gut says avid readers (which are always your #1 target due to word of mouth etc.) are generally fairly likely to pay for books.

I also once saw an author online who had a similar sentiment and offered a "did you download this book somewhere" button where you could donate a freely chosen amount of money to "ease your mind" and support future releases that worked rather well (to his own surprise). Trying to think of the name but can't I'll edit if it pops into my head.

<comment><USER11> Thanks for the feedback.

My book is neither tech nor mass-market, and I've heard from another author in the area that "piracy" doesn't seem to be a major factor. It's also a large enough file to be a pain to email or put on a free blog, and unlikely to turn into a reliable torrent, so accidental or casual mass distribution is unlikely. Hm...

</comment></comment></comment></comment><comment><USER13> Forget all the tooling. There are two battles:

1\. To write the best book you possibly can

2\. To print, market and sell that book

Far and away the most important thing is No. 1, because No. 2 cannot happen effectively without that. And the bar is high - very high. So in many ways the battle is lost or won by the time you've put down your pen. Yes marketing and distribution are incredibly important. Design too. But none of it is going to do much good unless the book really works. And making a book work is not hit and miss. It's the product of a very exact type of knowledge and skill - <USER2> is especially true of genre work. Of course, there have been a few tremendous exceptions - but you'll know if you're one of those.

So with self-publishing, as with ordinary publishing: first write a great book.

For that the tools you need are:

1\. Someone who can give you tough feedback and who knows what they're talking about

2\. A writing group for softer, more regular feedback

3\. Long practice in writing shorter work

4\. Long practice in critical reading of other people's work, especially unpublished work. You'll never make a mistake once you see that mistake and understand it in someone else's work.

<comment><USER14> There is a danger that any nuance you've intended accompany your main point may be lost, since this is indeed how a lot of technical people think: "If you build it, they will come", meritocracy, etc. It is the reason there are a lot of great, undiscovered products out there. However, marketing is the biggest piece of financial success (this may not be the chosen goal). Pretty much all non-technical activity can be classified as marketing except for required business paperwork. Writing something is literally the starting line.

Proof that " _none of it is going to do much good unless the book really works_ " is not reliably true, by counter-example: [https://www.google.com/#q=bad+books+bestsellers](https://www.google.com/#q=bad+books+bestsellers)

Random technical example (fast NTFS filesystem metadata search) I almost lost because I forgot to bookmark it after reading the only HN comment I've ever seen mention it: [https://news.ycombinator.com/item?id=13738309](https://news.ycombinator.com/item?id=13738309) And further, I just found the child comment sharing another utility!

Edit: Focusing more on preventing misinterpretation.

<comment><USER13> FWIW I work as a fiction editor for a publishing house. I only mention that because I'm really not that a technical person. I'm more on the arts side.

I accept that some bad books succeed, but this really is vanishingly rare. Subjectively even many so-called bad books have really excellent qualities - especially in narrative structure.

So there are Bad Books and then there are BADly written books.

I agree also with your point: build it and they will come doesn't work. Marketing is essential. But ask people to swallow an unreadable book - <USER3> is most books people write - and all the marketing in the world won't save you.

<comment><AUTHOR> I definitely agree with your initial post, I'm just not able to offer advice on how to write a good book as I may, or may not, know how :)

In the writing groups I've participated in, there seem to be a number of authors that are scared off by the process of getting their work out there. I'm hopeful that this post will help them overcome the technical hurdles in doing so.

<comment><USER15> There is a book out there called "Writing the Breakout Novel" by Donald Maass. The book was a tremendously help for me in deconstructing my plot and characters. I rewrote and even rearranged entire chapters. I had subplots that had no closure that I rewrote to provide an ending.

I invite you to check out my novel's Amazon page and read some of the reviews. Even the negative reviews praise the story itself, while slamming my writing style :)

<comment><USER16> Would you mind answering a few questions? I'm curious how you published your novel (self vs traditional), how you marketed, what your sales were like, etc.

<comment><USER15> Yeah no problem. Find my contact info at timecrossers.com

</comment></comment></comment></comment></comment></comment></comment><comment><USER17> How to self-publish a _paperback_ novel, that is. Looks like a lot of good advice!

For my use case (self-publishing a digital-only book):

I wrote "Learn Java the Hard Way" using Leanpub and I was pleased with the experience, but their build tools are closed-source and sort-of creaky and the tweak-compile-preview cycle is WAY too slow for my workflow.

I intend to use Softcover for my next one.

Recently I switched to Gumroad for fulfillment and I have been _incredibly_ happy with them. Highly recommended.

<comment><AUTHOR> Does Gumroad have a solution for VAT? That was one of the things that lead me to use Amazon instead.

Happy to update the post with alternatives.

<comment><USER18> >Does Gumroad have a solution for VAT?

Not the person you asked, and haven't tried the feature out myself, but saw this a while ago:

How We're Handling VAT at Gumroad: [http://blog.gumroad.com/post/110080508463/vat](http://blog.gumroad.com/post/110080508463/vat)

Here is an excerpt from that page:

[ We've talked a lot in the past about how we approach building Gumroad: We ask ourselves what creators are spending time on that isn't making things, and then we figure out how to do those things for them.

We are tackling VAT in the same way. Going forward, this is what creators on Gumroad need to do to properly handle VAT for their digital products:

Go back to making awesome stuff.

In other words, we're on it. Gumroad will collect VAT as required and remit it to the EU. You won't need to fill out any forms, register for anything, or send anything out. Your (EU-based) customers, will see (and pay) the added VAT on their purchases. ]

:)

</comment></comment></comment><comment><USER19> My wife is an aspiring novelist (on her 9th draft of the novel) with an MFA in fiction and some experience working for a literary agent. Some perspective from that side:

\- 99% of all drafts sent to literary agents are objectively non-publishable. Simple lack of quality, lack of maturity, or other quantifiable factors. One or two drafts is not enough, even for experienced established novelists. Sadly, you can often tell within the first 5 pages, and even within the first paragraph.

\- Out of the 1% that have the potential to be publishable, at least 4 drafts, and often much more, are required to reach the quality desired to actually be publishable.

\- Good novels that are simply not what the agent/publisher is looking for at that time is also a factor, but a very distant one relative to just quality of work.

\- In fact, if your novel is "good" but the agent/publisher is not inclined to pick it up, they are more inclined to ask for a full draft, to give great editorial notes, or to give referrals to other agents/publishers.

\- If an agent thinks that they cannot take a specific "Good" book to market, they will frequently give pertinent feedback and often suggest specific revisions in order to make such a book marketable.

\- At PNWA and other literary conferences vast majority of attendees bring single or maybe two-draft.

I should note that "number of drafts" is not an absolute requirement. Different folks write differently, and having a high number of iterations on your novel is not an indication of quality, rather than indication of prerequisite work required to produce a good product.

So while the publishing world is far from perfect, and both publishers and agents tend to gravitate towards what's fashionable, the reality is that the vast majority of aspiring work is simply far from finished, despite the authors' claims (this doesn't preclude garbage like "50 shades" from seeing the light of day, mind you).

Self-publishing gives those 99% a window to self-publish with only marginal quality controls. This in a way has the potential to overwhelm the system, and the objectively higher quality works can get drowned out in the noise.

Not to mention that, once self-published, a book is highly unlikely to be re- published by a traditional publisher, especially for a first-time author.

I'll close this tirade on a positive message. If you're a writer, you've already succeeded. The inner battles fought every single day for months and years on end alone make you a winner.

<comment><USER20> > One or two drafts is not enough, even for experienced established novelists.

 _cough_ Elmore Leonard _cough_

 _cough_ Robert Heinlein _cough_

> Not to mention that, once self-published, a book is highly unlikely to be > re-published by a traditional publisher, especially for a first-time author.

That's a good thing. I think more writers should be in a "fire an forget" mode, i.e write a book, self-publish it, have readers or not, and move on to the next story. Rinse and repeat until you are able to write good stories. I think it's much better for self-esteem to be able to think "I already wrote 5 novels. Most are shit but, hey, it's getting better and better every time" than "argh, I have to write that first novel from scratch, for the fifth time. I'm fed up with it. I've been working on it for 10 years. I feel like I won't ever finish it. Maybe this writing thing is not for me".

<comment><USER19> If you can write a great novel in one or two drafts, more power to you. I mentioned that it's not an absolute rule, especially for established experienced authors.

However, if you're an aspiring novelist, and you are frustrated that agents are not responding to your query letters with 5 pages of manuscript, and you've only done one or two drafts, there's a good chance that your book just isn't ready.

The other point I make is that the self-publishing industry allows for books that are objectively "not ready to enter the market" to enter the market, and at a ratio that overwhelms books that ARE objectively ready. This skews the market.

<comment><USER20> Yes, that's true, but I think self-publishing authors (at least on the ebook market) adapted. Amazon is very good on this market. More often than not, if you want to make money in that niche, you better publish series with a free first episode. If your book sucks, the reader will put it back very fast. Nobody will buy your series, and your book will quickly get lost in the abyss of amazon's search engines.

So, it kinda works. People who know you will be able to download your awful book, while innocent anonymous amazon customers will never find it by accident.

> However, if you're an aspiring novelist, and you are frustrated that agents > are not responding to your query letters with 5 pages of manuscript, and > you've only done one or two drafts, there's a good chance that your book > just isn't ready.

Yes, but the question is : is it worth keeping working on that book or is it better to forget and start working on a new one ? There is no definitive answer, but abandoning forever the first novel I ever wrote was the best decision in my writer life. YMMV.

</comment></comment></comment></comment><comment><USER21> The copy editor is a sore point with me. It's getting more and more common for me to run across jarring errors in novels. I don't mind so much if it's a digital-only 99 cent Kindle book, but if I spend six bucks on a book I don't expect to see duplicate sentences or incorrect word choice. "Tow the line", for example, or "then" when "than" would be correct.

<comment><USER22> > "Tow the line", for example, or "then" when "than" would be correct.

My favorite recent malapropism, seen more and more often and in increasingly exalted locales, is "reign it in". "Reign" is what a monarch does to a kingdom, "rein" is what a cowboy does to a horse.

</comment></comment><comment><USER23> I'm not sure that figuring out the tooling for indie publishing is too difficult these days. I recently published my first novel, and while it actually _isn 't_ self-published--it was accepted by a small press I'd worked with before--in terms of the tooling, it's being approached as a self- published project: I did the typesetting myself, for both ebook and print. (Like Zack Hubert, I used Vellum for the ebook, although I did the print version with LaTeX, which I'll probably write up a short guide to sometime.)

While my press paid for the cover art, printing, ISBN, and other stuff, and they've done some advertising, a lot of the promotional work is left up to me...and in a lot of ways I think marketing is a much harder problem to solve. My novel came out of an intensive workshop led by a Nebula, Hugo and World Fantasy Award winner (Kij Johnson), has a blurb from a recent Nebula novel nominee (Lawrence Schoen, author of BARSK), was available for pre-order, and I've done what I could to promote it...and it turns out all those people saying it's tough to get your book noticed are, surprise, absolutely right.

tl;dr: while I'm interested in how Mr. Hubert produced his book, I'm more interested in how he finds an audience for it. (Beyond writing an article about it that gets linked on the front page of Hacker News!)

<comment><AUTHOR> I'd love to read about using LaTex. Even though I've read TAOCP (shock!) and been a physics grad student, I'm still semi-scared at the notion of using it for fun.

Most of my grad buddies (in experimental physics!!) had endless challenges with it.

Surmountable, of course, but I wanted to aim for an easier skill set.

</comment></comment><comment><USER24> My advice would be to look at the pricing of CreateSpace versus Lightning Source/Spark before choosing one. And keep in mind that LS is the only one which lets you control the Ingram discount, down to 20%.

Here's the LS pricing guide: [https://myaccount.lightningsource.com/documents/LSI/files/pr...](https://myaccount.lightningsource.com/documents/LSI/files/pricing/USPricing.BW.pdf) and here's the CS pricing: [https://www.createspace.com/Products/Book/Royalties.jsp](https://www.createspace.com/Products/Book/Royalties.jsp)

For example, I sell a very short book via LS. The MSRP is $2.99, I set a 20% Ingram discount, and the print charge is only $1.56 (though the pricing guide now says $1.72; I think it's increasing soon). So I make $0.84 per book. With CreateSpace the lowest possible print charge is $2.15 per book, and given their fixed 40%/60% sales channel fees, it wouldn't be possible to sell the book for as low as $2.99.

LS isn't always cheaper though; you should do the calculation for your particular book size and page count.

Working with LS does make things a bit harder. You have to buy your own ISBNs, spend time waiting for certain manual processes (like for them to review your account application or a new book), and deal with their clunky website. But for me, it's worth the effort to be able to publish short books much more cheaply.

</comment><comment><USER25> If you plan on translating or writing in another language beware of KDP!

Time for my annual rant on KDP and Amazon not caring about writers in non- English.

The list of supported languages is incredibly limited: [https://kdp.amazon.com/help?topicId=A9FDO0A3V0119](https://kdp.amazon.com/help?topicId=A9FDO0A3V0119)

Finnish is allowed but Estonian is not... it is ridiculous. Latvian and Lithuanian is not allowed, Russian is not allowed etc etc.

It took a shaming campaign by the British press to get Welsch added.

This restriction is so silly when the book comes out perfectly on Createspace paper version.

It looks fine doing my own conversion for the Kindle but Amazon will not let me publish the books in unsupported languages.

I do editing and typesetting for a non-profit as a hobby/volunteer effort and end up publishing the paper books on Createspace but ebooks have to go through Kobo and other non Amazon venues.

</comment><comment><USER26> NOTE: _Step 2: Write the Novel ... You know, the easy part._

For those who have never written a novel, writer is being sarcastic here. The equivalent in terms of reference more familiar to the regulars on HN might well be: "Step 2: study for a CS degree, then decide which industry you intend to disrupt, learn how it works, and come up with a strategy. Oh, and write the killer app."

This is just the framework for the business plan: it's helpful, but it's insufficient on its own.

<comment><AUTHOR> Deeply sarcastic indeed!

Writing Singular was probably the most emotionally trying and, at the same time, rewarding challenge I've tackled. It even feels weird to describe it that way, it's simply been a part of me for the last two years.

I'm curious how that changes from the first book to the twenty-first?

<comment><USER22> > I'm curious how that changes from the first book to the twenty-first?

For your answer, compare "The Right Stuff" (1979) to "I am Charlotte Simmons" (2004), both by Tom Wolfe. The second book, apart from being unreadable, seems not to have been written by the same person.

</comment></comment></comment><comment><USER27> I self published a book of my poetry on Kindle Direct publishing 5 years ago see [http://a.co/iw4r1jq](http://a.co/iw4r1jq) I also used lulu to make a physical copy. For lulu I think I used Microsoft word and for kindle you could just make a HTML document to be published . The tooling in this article is great but as other people have pointed out marketing your book is more important. The article should be titled how to typeset a novel in 2017.

</comment><comment><USER28> I think the author missed the most important point (maybe they haven't gotten this far yet): marketing and _especially_ market-fit.

If you just want to release an ebook or even in paper, you can do that easily. If you want to release an ebook and have it be a success, that's _really hard_. It's hard even for traditional publishers, and it's certainly not any easier for self-published authors.

Anecdote: I helped my grandmother release her first book (here: [http://carolynnslaughter.com/book/](http://carolynnslaughter.com/book/)). The technical aspects are frankly not that hard. For anyone with halfway reasonable technical skills, launching a book is simply a matter of following various instructions online.

How much did we sell? Pretty close to rounding error of zero. Honestly, we were never expecting much success, because the book is fairly academic.

Edit: Less anecdotally: "Ninety percent of your book's success will be determined by the quality of your book. The other ten percent is distribution, marketing and luck." (From Mark Coker, founder of Smashwords: [https://www.smashwords.com/books/view/145431](https://www.smashwords.com/books/view/145431))

I understand that as HN readers we gravitate towards technical solutions. But the sibling comment by scandox is on the nose here. The hardest problem by far is writing a book that people will be willing to pay money for. The second hardest problem is marketing it, and all these other technical aspects come somewhere after that.

<comment><AUTHOR> I think there are many reasons for writing, only some of which are success/financially motivated.

Personally, I wanted to write a story about Lisp machines, Bostrom-inspired Artificial Intelligences, and a journey of discovery/adventure in the vein of my favorite stories (www.zhubert.com/reading). Not everyone gets excited about Lisp machines, so I'm not expecting a mainstream market.

That said, marketing was out of scope of my original article and quite a challenging problem, I agree.

</comment></comment><comment><USER29> CreateSpace is fine as self-publishing /print on demands goes, but just FYI another option is Lulu.com.

Disclaimer: I used to work for Lulu, but no longer do, and have no financial stake in them. Just wanted to point out the existence of another option.

</comment><comment><USER22> Quote: "TL;DR: Scrivener, Vellum, KDP, Createspace"

Scrivener is US$40, Vellum is Mac-only and US$200, the other two are unnecessary/replaceable and IMHO should be a list of equivalent options -- once you have a manuscript, which can be created using any number of free tools.

Someone who imagines writing the Great American Novel, but who also expects to have to pay a premium for each step along the way, may not grasp the essence of novel writing -- how it fits into the big picture.

There are any number of free word processors able to organize a book project into chapters and sections. Self-publishing is also free or should be.

My favorite story about novel creation is that of Andy Weir and The Martian. Weir started the project as a series of blog posts, got very useful feedback from his readers, improved his content on that basis, and the project grew nearly on its own. By the time Weir wanted to talk to publishers, they already wanted to talk to him.

<comment><USER23> To be fair, he's describing the tools that worked for him. Scrivener is for taking stories from notes through first draft; it's not a word processor in the same way Word and LibreOffice are, and there's no direct free software equivalent. You could use free tools for the same tasks, but you'll do those tasks differently--and that might be preferable to you, depending on how you work. Scrivener is very powerful, very flexible, and very idiosyncratic.

Vellum is similar, but there _are_ direct free software equivalents, most notably Sigil, a GUI ePub creation tool. Alternatively, you could use Pandoc to assemble an ePub from Markdown (or any of the seemingly several thousand other markup dialects Pandoc knows). I ended up using Vellum for my book because it was incredibly smart about taking a DOCX file exported from Scrivener and turning it into a properly sectioned and tagged ePub, and adding new sections, artwork, metadata and overall book styling is trivially simple. Sigil and Pandoc are great, but they're also lower-level tools by comparison, with both the advantages and disadvantages that implies. For me personally, it was worth spending $30.

 _Self-publishing is also free or should be._

Well, it's as free as you want it to be. In my experience, YMMV and all other appropriate disclaimers, producing an ebook which seriously competes with ones from traditional publishers requires skill not just in writing but in editing, graphic design, typesetting (or ebook production), and (depending on the cover) illustration. If you have access to all those skills without spending any money, awesome, but it's kind of a tall order.

<comment><USER30> Scrivener now spits out epub and I believe mobi formatted final products if you wish. Use is non-trivial however.

</comment></comment></comment><comment><USER31> This is not related to the article, but US-based HN readers may benefit from it: You can set up an account with your local library to 'check out' books for your Kindle completely free. If you an avid reader like me, this saves a lot of cash in buying those books and reading them once or twice. There are some restrictions on the number of titles and the length of 'check out' though. Each library has different rules and stipulations. Check out if your local library has this program, most cities do by now.

</comment><comment><USER11> Interesting. I recently published a paperback in a different space (color, lots of figures), and made completely different choices. I can't write anything nearly as long or specific as your blog post here, but in brief:

* I used XeTeX plus my editor of choice.

* I sent it to a few friends, listened to what they said, and read and re-read until my eyes bled.

* I avoided Amazon like the plague, instead using a small print-on-demand shop with reasonable prices.

* I sold via pre-sales, my personal website, and consignment at small bookstores.

</comment><comment><USER32> The article addresses the finishing rites for self-publishing, but the note on tl;dr at the top should be expanded to say something like: writing a novel is the equivalent of starting a small company. It takes at least three months of focused effort to write something worthwhile for an experienced novelist, a year or ten for a novice.

Also, it might be helful to note tha the likelihood of making a living from writing is dependent on how fast an author can complete each novel.

</comment><comment><USER33> Shout out to @zhubert, cool article -- thanks for sharing it! This is how I'd write a novel in 2017 and do it for one web instead:

[https://blog.marvindanig.com/2016/03/17/how-to-write-a- super...](https://blog.marvindanig.com/2016/03/17/how-to-write-a-superbook/)

Disclosure: I'm the developer/engineer.

<comment><USER34> This is very interesting. Does it use markdown as well or just plain markup/html in the manuscript?

</comment></comment><comment><USER35> If you're not of the "I'm writing because I can't not" personality, you might also want to consider self publishing book statistics.

[http://www.bowker.com/documents/self-publishing-report-in- un...](http://www.bowker.com/documents/self-publishing-report-in-united- states-2008-2013.html)

<comment><USER36> Not sure what point you'd like to make with those statistics. That a lot of books are written? Probably more interesting to a prospective author would be statistics on how many of them are read.

<comment><USER35> Approaching 1 million annually.

That's 1 per 317 Americans.

</comment></comment></comment><comment><USER37> Very nifty guide and honestly very needed in the DIY space I think. Mostly because I feel long-form print (specifically hard copy) is occupying a smaller and smaller market share year after year. There's definitely still demand - <USER4> in airport bookstores - but the ROI doesn't seem very appealing.

My experience was through Smashwords and I was pleased with the platform. Never even recouped the $10 or so fee to get the ISBN number hah. It's a nice opportunity - self publishing, that is - and I think lots of amateur writers should feel compelled to give it a try. Nothing is quite like the artistic experience of working very hard on a piece, getting it out into the world, and almost nobody paying any attention to it. Except a couple purchases by relatives.

Not everybody gets to be Chuck Tingle, which, while fiscally successful, is the literary equivalent of the contents of a mop bucket at a seedy peep show joint. Ugh.

<comment><USER38> > I feel long-form print (specifically hard copy) is occupying a smaller and > smaller market share year after year.

Actually, after several years of decline, print sales have been bouncing back in the past couple of years:

[http://www.publishersweekly.com/pw/by-topic/industry- news/bo...](http://www.publishersweekly.com/pw/by-topic/industry- news/bookselling/article/72450-print-book-sales-rose-again-in-2016.html)

</comment></comment><comment><USER39> Nice guide, although it is more of a 2016 guide. Amazon recently launched the public beta of KDP Print making CreateSpace redundant.

[https://kdp.amazon.com/help?topicId=AH8RA6CMVRN8Y&ref_=pe_29...](https://kdp.amazon.com/help?topicId=AH8RA6CMVRN8Y&ref_=pe_2983330_227202760_kdp_BS_D_pgs)

<comment><AUTHOR> As soon as they have author proofs, I'll switch over. The interface of KDP is really nice, trust me, I can't wait to switch.

</comment></comment><comment><USER40> Scrivener exports to Word, and various other formats. Working with large manuscripts in Word is a challenge, and it lacks many of the writing features of Scrivener. I've personally never had a problem with Scrivener's Word exporter, and have never felt like I needed to purchase Word.

<comment><AUTHOR> Largely driven by my copyeditor only working in Word. Given that incorporating their recommendations was much easier with change tracking in Word, it was a very useful addition to the toolbox.

</comment></comment><comment><USER41> > The lingua franca for most writing is Microsoft Word so you'll have to buy > that eventually...

I don't understand why something as ubiquitous as writing necessitates spending $100 on software from Microsoft. A plain text editor works out of the box. If you have a need for more complex formatting you could use LibreOffice, OpenOffice, Pages (comes free with a new Mac/iPad), WPS Office, etc. I get that Word has is the standard and all, I just don't understand why the agreeded upon standard needs to cost upward of $100, especially for something as simple as word processing.

<comment><AUTHOR> It wasn't the formatting actually. Change tracking of the proposed edits from the copyeditor was greatly expedited through Word's toolset. While I could have copied them all over by hand, I figured I'd likely introduce other errors in the process. Clicking accept/reject was worth the peace of mind.

<comment><USER42> It's my (possibly false...) impression that those who find Word's "Track Changes" valuable, have never used diff/patch or any of their many descendants (i.e. git etc.) Trying to keep _two_ contributors' work straight with Word's "Track Changes" is a hair-pulling challenge. With more people, forget about it. In large groups, I've found even "change the third sentence in the ninth paragraph" requests via email to be better...

<comment><AUTHOR> I can't address the general impression, but as someone who spends nearly every moment of my day in git (and only twenty percent of my day dealing with conflicts, I jest) I think my advice is more towards impedance matching.

When working with folks outside of tech, something like Word expedites the process as it's their native tongue. While tools like diff and patch are great (I was a Unix admin at Amazon back in the Jurassic period), they'd limit my options for copy editors.

</comment></comment></comment></comment><comment><USER43> Literature doesn't work this way... It's hard to get your first book published because you are unknown in the literary circles. One has to start with introducing themselves in literary journals (or even blogs), publishing short stories, reviews, essays, etc. Probably none of the acclaimed authors started out with a big novel. Then, publishing with a known editor is easier, and has many positives: their name alone helps increase the amount of potential readers.

</comment><comment><USER44> Thanks for this, it's interesting to see an article that goes from showing step zero all the way to selling on Amazon. I tried getting my novel published in 2012 and was rejected by many literary agents to the point I was having trouble finding any left that I had not submitted to. I considered self- publishing at the time but decided against it. I'll keep this bookmarked should I choose to self publish in the future.

<comment><AUTHOR> I'm happy that this might to useful to you in the future!

</comment></comment><comment><USER45> I've been toying with the idea of a site for original written works, in the same vein as current fanfiction sites or Deviantart.

I imagine it would end up working along the shareware model, x chapters here for free and purchase access to the full story, or something along those lines - though if there is a market for it you could use it as a streamlining process for publishing houses I suppose

Is anybody aware of a site like that in the wild already?

<comment><USER46> Not with your proposed marketing mechanism, but there's [https://www.fictionpress.com/](https://www.fictionpress.com/) \-- a spinoff of [https://www.fanfiction.net/](https://www.fanfiction.net/).

</comment></comment><comment><USER47> As someone who is working with another chef to release their own cook book and associated app, this and the comments are immensely helpful. Thank you :-)

<comment><AUTHOR> Hurray! That's fantastic!!

</comment></comment><comment><USER40> One thing I would suggest, which isn't mentioned in the article, is to join a writing group. It will give you an opportunity to both give and receive constructive criticism, see what works and what doesn't, and will help improve your work overall (if you listen to the criticisms, anyway).

<comment><AUTHOR> Excellent advice. I tried to stay focused on the technical side of production in this post as reams could be written on how to improve the writing side.

</comment></comment><comment><USER48> I would add to the start of this:

prerequisite: Mac computer

I've recently self-published using Windows / Linux and didn't have these tools available.

I will say that Createspace is an awesome company. I just think its brilliant I can upload a PDF, and in a few days I can buy 100 copies of it on Amazon, and have it shipped same day.

</comment><comment><USER49> I'm curious. Do you jot everything down on Scrivener? Or do you also write on paper?

Lately I've been carrying a notebook everywhere I go and I'm starting to write much more.

<comment><AUTHOR> Both. Ideas for the story accumulated in a Moleskin for a few months before I started writing. I also carry around a little field notes booklet in my backpack/pocket a good bit of the time. Makes me feel like a creative :)

The major edit was done via 3x5 cards. Each card had a scene, the values it turned on, and the actors in the scene, so I sort of treated it like a screenplay. 3x5 cards are really nice for juggling things around (though Scrivener does has a corkboard view, I liked the paper approach for this step).

</comment></comment><comment><USER50> If your scifi book synopsis contains one of the phrases:

1\. journey to the end of the universe

2\. the fate of the universe hands in the balance

3\. voyage to the end of time

I put the book back on the shelf.

<comment><USER51> Only one man can save the fate of mankind...

<comment><USER50> With thrillers, they synopses all sound the same. A grisly murder, a dark secret, and one person must follow the clues to find the murderer before he kills again. Zzzzzz....

</comment></comment></comment><comment><USER52> I have some of my books on Amazon KDP and a few not. KDP is nice because it's easy. But, the exclusivity can suck.

<comment><USER53> You are mixing up terms.

KDP refers to the entire publishing program.

KU, Kindle Unlimited requires exclusivity but you get paid by page read.

<comment><USER30> He may be referring to using KDP's freebie program, that requires exclusivity. I keep waiting for Amazon to be dinged for monopoly abuse on that policy, but I'm still waiting.

</comment></comment></comment><comment><USER54> More or less off-topic question. How to not get sued by overzealous copyrights holders?

I have an idea for an informal Chemistry 101 notebook. I might end up making comparissons between cartoon characters and periodic table elements: Hydrogen ~= Tinkerbell, Hellium ~= Master Yoda, Lithium ~= Philoctetes (from Hercules movie), etc.

It is not reassuring that most references in that list are owned by Disney Co.

<comment><USER55> >How to not get sued by overzealous copyrights holders?

I might be misunderstanding, but why is it unreasonable for the holder of a copyright to object to an author using their work without licensing it? This in particular seems like a pretty clear-cut example of actual infringement.

<comment><USER30> Not sure why you don't see the mere mention of cartoon characters as lying outside fair use. [https://en.wikipedia.org/wiki/Fair_use#1._Purpose_and_charac...](https://en.wikipedia.org/wiki/Fair_use#1._Purpose_and_character_of_the_use)

<comment><USER55> Oh, I'd readily admit _mentioning_ a cartoon character is fair use, but it seems like using the _actual image_ of a cartoon character is infringement—what else IS a cartoon character? Where's the line? If one person can put Tinkerbell in a science book, why can't I publish a "Disney characters coloring book" without a license?

I'm admittedly not an expert, but there have been many rulings stating that characters themselves can be covered by copyright,[0] and fair use is supposed to allow for commentary and criticism, not just "oh, that's nice, I'll use that." It's not very different from just putting people's photographs in a science book, is it?

[0] [http://fairuse.stanford.edu/case/dc-comics-v- towle/](http://fairuse.stanford.edu/case/dc-comics-v-towle/)

<comment><USER56> > Oh, I'd readily admit mentioning a cartoon character is fair use

It's probably not even an otherwise-infringing activity that would require fair-use analysis. At least if we are talking about copyright; if the character name is also a trademark, that may be a different story.

</comment></comment></comment></comment></comment><comment><USER57> Prolifiko are doing a great job of supporting writers via its app: [http://www.thememo.com/2017/01/04/prolifiko-writing-tool- pro...](http://www.thememo.com/2017/01/04/prolifiko-writing-tool-prolifiko- kickstarter-publishing-news-write-track/)

</comment><comment><USER58> Seems I am on a downvote streak lately so I'll keep going; why would you not use (Xe)LaTeX? If I was too lazy to use that I would probably use something like bookdown.

<comment><USER59> I am using LaTeX, but I'm almost certain it will be a pain when I go to the eBook step, since they all have their own formats and I'm not aware of any tools that can render the math formulas in my book faithfully on all of them.

<comment><USER60> This is a problem. In epub, you can use MathML, but support on reading systems is spotty. You could embed something like MathJax, but that would also depend on there being scripting support (not all systems have it). On the Kindle, there is none--no Latex, no MathML, no scripting, nothing for math support at all. Your only choice is graphics. SVG or high-res PNG works best. Though on a Kindle, there's no "S" in the SVG (i.e, the Kindle can't actually scale SVG). Occasionally we do a math-heavy econ book here, and it is always a compromise.

<comment><USER59> What is your opinion on publishing a math-heavy e-book (like, equations on every page, inline and offset)? I don't think I will be able to do better than having a physical book paired with a pay-to-download PDF.

<comment><USER60> I guess it comes down to what the market is, how much work it would take to do the conversions, and what compromises you're willing to take. Inline math is easier to deal with, usually it's just a matter of making sure the characters display correctly. Complex multi-line math is where things start to fall down. If this is a specialized technical reference that is of interest to 500 people, it might not be worth a lot of work to sell 20 copies on Kindle.

</comment></comment></comment></comment></comment><comment><USER61> tldr:

\- use proprietary tool №1

\- enslave yourself to Amazon

\- use proprietary tool №2 from Amazon

<comment><USER62> Others offering self-publishing advice stress using both Amazon CreateSpace and IngramSpark/Lightning Source for broader distribution.

See: [http://selfpublishingadvice.org/watchdog-ingram-spark-vs- cre...](http://selfpublishingadvice.org/watchdog-ingram-spark-vs-createspace- for-self-publishing-print-books/)

<comment><USER33> Not everyone wants to live by amazon or even succeeds with it.

Try [https://bubbl.in](https://bubbl.in) for example if you wish to sell your work like a responsive book straight on the web. Zero lock-ins and great sales outside of a garden if you will ;)

Full disclosure: I'm the developer/engineer behind the service.

</comment></comment></comment><comment><USER63> Ugh Facebook really? I would want t read it but how do I remember? An email would work maybe. But a text would get me for sure but I don't have Facebook

<comment><USER64> I don't understand, I didn't see any reference to Facebook aside from the standard social media links at the bottom.

<comment><USER65> I think he's referring to the "book club" on [http://www.singularbook.com/](http://www.singularbook.com/) which is a link to Facebook.

That homepage also has a 3MB image of the cover on it, bit excessive. Google reckon on saving 2.8MB by image optimisation (or 3.4MB when targetting desktop), [https://developers.google.com/speed/pagespeed/insights/?url=...](https://developers.google.com/speed/pagespeed/insights/?url=http%3A%2F%2Fwww.singularbook.com%2F) .

<comment><AUTHOR> I created the Facebook group for readers who are reading the novel as its released serially online (for free).

Each week I release three more chapters on the website (www.singularbook.com), not on Facebook. It was just something to do for fun :)

Yep, the cover image is a bit excessive. My gulpfile didn't have that image in it but I'll get that sorted out.

</comment></comment></comment></comment><comment><USER66> Talking arty, literary novels are form more than content, so why self-publish a literary novel if form is strong enough?

<comment><USER54> Not sure what you meant, but I think an appropriate answer would be: for fun and profit.

It is a bit like saying, why go climbing the mount Everest if lots of people have already done so. Indeed, why trying to do stuff nobody else have done before, exclusively? If billions of people have already existed before yourselves, all the good stuff worth doing will have been done over and over again, and the only original stuff left will be JackAss(TM) material.</comment></comment>


Show HN: Nuevo Cloud - File Storage using WebDAV - <AUTHOR> https://www.nuevocloud.com

<comment><AUTHOR> I built this solo.. it took about a year to get it this far. Feedback is appreciated.

I wrote the filesystem, webdav server, and a web framework (used for the website and webdav server).

Written in C... and it runs on Debian + Apache + PostgreSQL

The filesystem has support for dedup, encryption, end-to-end checksumming, and a few other features. Copy-on-write... so it keeps a complete history of changes. It uses the userid + filename + date/time to reference files, so the complete history is accessible. The history is controlled by the "erase delay" setting right now, but I plan to create a nicer interface for it later (so you can browse your files at any point in time).

When files are uploaded, control is passed from the WebDAV server to the fs.. so files are encrypted as they're uploaded.

So what do you think?

<comment><USER2> The most important question: can you, the provider, see/decrypt any data?

<comment><AUTHOR> I was waiting for this question... The encryption is done on the server.. so it is possible to decrypt the data. However, I haven't written any tools to do that; I certainly don't think I have any right to look at what you store, nor is it useful to me in any way; and I am serious about the ToS -- decrypting the data requires a valid subpoena.

Also you can use client-side encryption (which I cannot decrypt). Everything else will still work -- dedup, history, etc.</comment></comment></comment>


Show HN: lightweight Blog / Tumblelog engine "embr" - <AUTHOR>

Hey everyone,
For the past few weeks I've been working on a new lightweight blog engine powered by Codeigniter called "embr". I've stuck the source on Github and would love to hear any feedback or new features to add.
Features:
- Basic image and text posts
- Tag & Category support
- Disqus commenting
- Infinite-scrolling on index pages
- Tumblr style "reblog" is nearly complete (which will allow reflagging of posts across self hosted domains)
Check out the source on Github: https://github.com/laurencedawson/embr
The live demo can be viewed here (username: firstname.lastname@example.com pass: qwerty) http://blog.laurencedawson.com
Thanks <comment><USER2> Good stuff. If I can hassled into moving away from jekyll I'd probably migrate to this first.

I like Tumblr but the data lock-in + shoddy uptime scared me, and this uses CodeIgniter—something I'm familiar with—so I'd probably have an easier time hacking on it than even jekyll!

</comment><comment><USER3> Can't see tags/category showing up on a demo post, but it's looking like a really lightweight solution. Great work!

In your opinion, what would make someone use this over setting up a hosted Tumblr?

<comment><AUTHOR> Ah no posts in the demo have tags or categories but its coded up.

Once its all setup I'd like to think being distributed and having full control over your data would be a strong selling point. No more "tumblr is over capacity" messages.

</comment></comment><comment><USER4> Very cool. One question: how does google index the content that hasn't loaded yet? (the infinite-scrolling thing)

<comment><AUTHOR> The scroll degrades gracefully to standard pagination if a user doesn't have JS enabled

</comment></comment><comment><USER5> Wow, I have checked it, it looks cool, esp. the logo

<comment><AUTHOR> Thanks

</comment></comment><comment><USER6> Awesome Job Laurence :)

</comment><comment><USER7> Awesome job

</comment><comment><USER8> If I saw this before github/jekyll I would probably use it.

Does the software handle SEO which would be a selling point for me when considering a blogging platform

<comment><AUTHOR> Dang too late!

It's fast, by default everything is cached which I think helps with SEO. Also it generates meta + opengraph tags for each post, hyphens instead of underscores in URLs etc. Not sure what else to add, any suggestions?

<comment><USER8> Get some contact details up or your github page I am quite interested in your project.

<comment><AUTHOR> I've added my email and twitter to the main readme, would be great to hear from you.</comment></comment></comment></comment>


Show HN: Monocypher, my crypto library, has been audited [pdf] - loup-vaillant https://monocypher.org/quality-assurance/MON-01-report.pdf <comment><AUTHOR> I know next to nothing about crypto, but it sounds good :) Congratulations!</comment>


'There's nothing left': Crew demolishes wrong home, blames Google Maps - <AUTHOR> https://www.washingtonpost.com/news/morning-mix/wp/2016/03/25/theres-nothing-left-crew-demolishes-wrong-home-blames-google-maps/?tid=sm_tw <comment><USER2> > The company's slogan, according to its website, is "We could wreck the > world."

What a slogan.

<comment><USER3> That could be a threat, just like "don't be evil"...</comment></comment>


Monstache: realtime MongoDB to ElasticSearch replication - <AUTHOR> https://github.com/rwynn/monstache <comment><USER2> Any advantage over mongo-connector? [https://github.com/mongodb-labs/mongo- connector/wiki/Usage-w...](https://github.com/mongodb-labs/mongo- connector/wiki/Usage-with-ElasticSearch)

<comment><USER3> connector allows one to select the fields which are replicated but monstache goes a bit further by integrating the Otto embedded javascript engine. This allows for modifying or adding fields in addition to dropping at the collection level. It appears connector's field option is global to all collections. monstache also has a stateful mode (using the resume flag) which allows one to replay only the events missed since the last run. Another alternative is transporter from compose which is also has the Otto engine and is written in Go. However, monstache is a single purpose tool while transport is a more generic tool.</comment></comment>


Wyze removing person-detection from cameras after Xnor terminates edge-AI lease - <AUTHOR> https://forums.wyzecam.com/t/an-update-regarding-person-detection/73107 <comment><USER2> Wow, that was fast. I wonder, what made xnor.ai terminate the contract? Signing with a competitor, or an exit?</comment>


Show HN: Morning Short Listen – Audible for Short Stories - <AUTHOR> http://listen.morningshort.com/ <comment><AUTHOR> Let me know if you have any questions. Happy to answer them - Michael - curator of Morning Short</comment>