How breakpoints are set - <AUTHOR> http://majantali.net/2016/10/how-breakpoints-are-set/ <comment><USER2> Also worth noting: x86 supports four hardware breakpoints, which are sometimes more convenient, since they don't involve overwriting any instructions. (They're also more flexible because they can be used to trap on memory reads, etc.) [https://en.wikipedia.org/wiki/X86_debug_register](https://en.wikipedia.org/wiki/X86_debug_register)

<comment><USER3> I thought that these HW 'breakpoint' registers were (only) used to implement watchpoint, that is, read/write operations on data. Can they be used to implement 'instruction breakpoints'?

<comment><USER2> You can set the "break on" bitfield to "execute" (0) to have the debug register function as a normal breakpoint.

</comment></comment></comment><comment><USER4> The reason INT 3 is used is that it's the only interrupt that has a single byte opcode (0xCC). Other interrupts require two bytes: CD <interupt number>.

This makes setting a breakpoint really easy, as all you have to do is replace a single byte (and restore a single byte) where you want to place your breakpoint. INT 3 being only one byte is also important when you're setting a breakpoint instead of a another single byte instruction - your newly set breakpoint won't override the consecutive instruction, which might be jumped to somewhere else in the code.

<comment><USER5> > The reason INT 3 is used is that it's the only interrupt that has a single > byte opcode (0xCC). Other interrupts require two bytes: CD <interupt > number>.

It's kind of the other way around. The reason it has a single byte opcode is because Intel wanted INT3 to be for break points, so they designated 0xCC for it. In fact, 0xCD 0x03 works, but just isn't used.

<comment><USER6> Because x86 instructions can cross 4/8/16 byte alignment boundaries, you can't safely set a multi-byte breakpoint in all cases. CPU might execute (bytes [0xCD, x] -> int x) instruction before the parameter becomes visible and trigger some other exception, whatever happened to be at that address before.

</comment></comment></comment><comment><USER7> _You can consider your debugger to be a program which forks() to create a child process and then calls execl() to load the process we want to debug_

That is one way to look at it, but I find it a bit too limiting (debuggers can attach to an existing process as well) and too confusing (requires knowing what fork does/is, same for execl - and are those even used when attaching to an existing process?) and because of the latter functions used obviously coming from a linux background (nothing wrong with that, on the contrary, but I can imagine windows people or beginners still having no clue whatsoever about a debugger after reading this - though it's likely not the target group).

<comment><USER8> There is also remote debugging. Via network or JTAG.

</comment></comment><comment><USER9> I went 20 years without knowing how breakpoints work because they always just did (when they were available). Reading this, it's unsurprising how the tool works. That's the best kind of tool.

<comment><USER4> What's even better is that breakpoints haven't really changed over that period of time. They just work.

While a lot of tech is rapidly moving and constantly changing - this is the type of fundamental knowledge that will probably prove valuable for the rest of your career.

<comment><USER10> Is this pretty much how all breakpoints work even in higher level languages with an IL/Bytecode etc.?

<comment><USER2> Usually they either run debugged code in the interpreter or recompile the JIT code in a special instrumentation mode. (Not having to monkey patch the code at runtime—being able to do a "proper" recompilation—is one of the advantages of having a JIT!)

See this explanation of how it works in SpiderMonkey, for instance: [http://rfrn.org/~shu/2014/11/20/speeding-up- debugger.html](http://rfrn.org/~shu/2014/11/20/speeding-up-debugger.html)

<comment><USER11> Specially when coupled with the possibility to live connect to a production instance and do all sorts of monitoring and reports.

Of course, one needs to take care of the respective secure access. :)

</comment></comment></comment></comment></comment><comment><USER12> Weird, I just happened to be reading about this topic last night. If you liked this, make sure to check out the articles by Eli Bendersky in the footer. He has a 3 part series on how debuggers work:

[http://eli.thegreenplace.net/2011/01/23/how-debuggers- work-p...](http://eli.thegreenplace.net/2011/01/23/how-debuggers-work-part-1) [http://eli.thegreenplace.net/2011/01/27/how-debuggers- work-p...](http://eli.thegreenplace.net/2011/01/27/how-debuggers-work- part-2-breakpoints) [http://eli.thegreenplace.net/2011/02/07/how-debuggers- work-p...](http://eli.thegreenplace.net/2011/02/07/how-debuggers-work- part-3-debugging-information)

</comment><comment><USER13> How would a program detect the use of a debugger? I know a lot of crackmes and other anti-piracy measures involve detecting the use of one but am not sure how they do it. Do they just look for running processes with a debugger signature like softice?

<comment><USER14> One technique I remember being used in DOS apps from many years ago was that the code would be encrypted in such a way that the next instruction to be executed would only be decrypted immediately before it was run. This was achieved by setting up the single step interrupt as the decrypter, and running the code in single step mode.

The fact that the code was encrypted meant the debugger couldn't disassemble it in any meaningful way, and also made it impossible to set a breakpoint (since the breakpoint would just end up being "decrypted" into some other opcode that would inevitably crash). The debugger also couldn't step through the code, because taking over the single step interrupt would prevent the decrypter from running, so you'd just be stepping through garbage.

The way I worked around this was by writing a debugger that could hook the single step interrupt in such a way that it still forwarded the interrupt onto the previous hook. I still couldn't set breakpoints, but I could step through the code, watching it decode itself as it proceeded.

<comment><USER15> If you're single-stepping you don't need to patch the instruction stream for a breakpoint; you can just test the instruction pointer against the break address at each step.

<comment><USER16> Not every processor has the means to perform direct comparisons against the program counter; as far as I know 6502 and MC68000 families can not do direct comparisons with the program counter.

<comment><USER15> It's the program counter of the state that was interrupted by the single-step trap that's of interest, and _that_ program counter is generally pushed onto the stack by the trap.

For example, this is how the trace exception on the M68k works - the program counter of the next instruction to be executed can be read off the stack by the exception handler. The 6502 doesn't have built-in software single-stepping but the same effect was sometimes achieved by tying a short timer to the NMI - and when the NMI is asserted, the interrupted program counter is pushed onto the stack.

</comment></comment></comment></comment></comment><comment><USER17> Nice explanation.

However, it didn't explain how the debugger can stop again at the breakpoint after the last step? The interrupt command has been replaced with the original command, so the process won't stop again..

<comment><USER18> Usually the debugger just replaces the breakpoint instruction with the original byte, then resumes execution in a single-step mode that causes the CPU to just execute a single instruction before firing another interrupt. After that, the debugger sets the breakpoint again and resumes normal execution.

<comment><USER19> When single-stepping, it's necessary to step only one thread, to ensure that other threads don't skip the (temporarily) disabled breakpoint. There's a paper here that discusses one solution to the problems this causes: [http://www.bmrtech.com/uploadfile/image/whitepaper/mentorpap...](http://www.bmrtech.com/uploadfile/image/whitepaper/mentorpaper_multcore_db.pdf)

(You can engineer a deadlock in gdb due to this, e.g., on x64, by stepping over a SYSCALL instruction that reads from a pipe that's about to be filled by another thread. But you're unlikely to experience this in practice, as system calls are wrapped by a glibc function, and you'll probably be stepping over that rather than the instruction directly.)

</comment></comment></comment><comment><USER20> Any comments on how breakpoints work on external targets, running bare-metal, and you can't replace instructions? Ex. debugging an 8-bit AVR, say atmega1280 over JTAG. I am guessing it has to do with the JTAG doing a simple compare of the PC with the breakpoint address, just want confirmation and more details.

<comment><USER21> yeah, processors such as the ARM cortex-M series have built in debug features such as ~8 breakpoint registers that trigger a change in processor state when they match a code or data access address.

The breakpoint registers are accessed via JTAG/SWD using your j-link/FET/whatever.

Quite often, when you're debugging embedded systems, you run out of "hardware" breakpoints and have to resort to software-style breakpoints described in the article.

<comment><USER22> Yes. And these software breakpoints are written into flash. This is why they're very slow - and why they wear down your flash.

The situation is kinda OK when you don't often change breakpoints and your CPU has an instruction register writable by the debug probe via JTAG/SWD/etc. Upon stepping or continuing from a breakpoint, the debugger will write the actual instruction at the breakpoint into the instruction register and tell the CPU "run again, but don't load the instruction from memory as I have already loaded it into your instruction register.".

Another option is to emulate the effects of the instruction in the debugger and write the results back into registers/RAM/I/O. This is not always possible.

If you don't have the options above, your flash will wear down quickly, as stepping away or continuing from a breakpoint entails writing the actual instruction back, stepping, then writing the breakpoint again.

<comment><USER23> Is "wearing down the flash" really still an issue? Hard drives these days are flash-based - surely the read/write cycles modern flash memory can withstand are high enough to manage a few breakpoints?

<comment><USER24> Yes. In an embedded environment, its not uncommon to:

1\. Have a NOR flash part rated for 10,000 or fewer cycles. 2\. Have no facility for remapping bad sectors. 3\. Have no wear-leveling mechanism.

All of this is reasonable for a product that will only be flashed once during manufacturing (and there are a lot of those products) or a product that will receive a firmware update a single-digit number of times in its lifetime.

In contrast to an SSD, where:

1\. NAND flash is used, with 100,000 to 1,000,000 write cycles 2\. The drive can transparently remap bad sectors, so flash can start to fail before anybody notices. 3\. The drive performs automatic wear leveling - if you try to write a single sector a million times, the drive will do something closer to writing a million sectors once.

<comment><USER6> > NAND flash is used, with 100,000 to 1,000,000 write cycles

Damn, where can you get that kind of flash chips with even 100k cycles endurance? I'd like to place a _large_ order. 64 Gbit chip, please.

1000-3000 P/E cycles is typical endurance for MLC NAND flash, not 100-1000k. SLC chips would fare better, but larger ones are too expensive for typical applications.

</comment></comment></comment></comment></comment></comment><comment><USER25> Others have mentioned hardware breakpoints but I also want to mention another type of breakpoint that can be used for both code and data and does not require program modification. And I can name at least one debugger that uses this method. The method is hooking Page Faults. Set the present flag of the page you want to debug to false and you'll be notified of any code that is executed there. But you can also be notified when data is accessed and by which instruction and address.

On Windows this can be done with SEH and Linux has its own thing too.

</comment><comment><USER16> 1\. contemporary central processing units have large instruction and data caches.

2\. ptrace() modifies the currently debugged program.

3\. if the program currently being debugged is sufficiently small, it might end up in the processor's instruction cache.

4\. instruction cache invalidation, at least on some processors like MC68000 family, causes crashes.

5\. since ptrace() effectively performs the equivalent of self-modifying code, how is instruction cache invalidation avoided?

</comment><comment><USER26> So when stepping through are debuggers just using breakpoints over and over? how is a step through different than setting a breakpoint if at all?

</comment><comment><USER11> No mention of hardware breakpoints?

<comment><USER6> Indeed, x86 hardware breakpoints are way more powerful and don't require changing instructions. Only problem there's just 4 of them, but it's usually enough. It's nice to be able to set a breakpoint on data access as well, it's often more useful than on instructions.

[https://en.wikipedia.org/wiki/X86_debug_register](https://en.wikipedia.org/wiki/X86_debug_register)

</comment></comment><comment><USER27> Fun tidbit.. AIX ptrace() doesn't support single stepping (PT_STEP). So the way that single step is implemented is by actually interpreting the instruction and if it is not a branch, then simply do the breakpoint swap mentioned in the post. If it _is_ a branch, then decode the instruction (for all the various types of branch instructions!), actually compute all forms of the implicit/explicit branch target (register based, offset based, absolute addresses, etc) and do the instruction swap at all the possible branch targets. After one of the branches is taken, then put everything back. Oh, and care must be taken to not disturb atomics...

[https://sourceware.org/git/gitweb.cgi?p=binutils- gdb.git;a=b...](https://sourceware.org/git/gitweb.cgi?p=binutils- gdb.git;a=blob;f=gdb/rs6000-aix- tdep.c;h=acd52bbd9548d6600a7246f8d984410bea72fc64;hb=HEAD#l670)

<comment><USER28> Why would you need to put breakpoints in more than one place? Presumably when you're about to execute the instruction, you know the values of all the registers and memory addresses so you can just interpret where the jump goes. What am I missing?

<comment><USER27> If memory serves, there are certain branch conditions that can't be easily queried via ptrace to implement it that way. So instead, you enumerate the targets and break on all of them.</comment></comment></comment>


Ask HN: Anyone running a US-based non-resident LLC? - <AUTHOR>

Is anyone running a US-based LLC with non-resident members? I'm thinking about registering it for my web venture. I did some research, but I cant find anything about few things..
Is it hard to do LLC book-keeping for web app  on my own?
If I hire myself as a contractor in my home country, do I have to pay taxes on these incomes to US gov?
If I get paid from US-based company personally (not through company), do I have to pay taxes to US gov on these incomes?
If my company gets money from both US and non-US units, do I have to pay taxes to US gov for both or US units only?
Is it hard to run a US-based LLC from abroad in general? Paperwork etc.
Thanks! <comment><USER2> You've touched upon a large and complicated subject but here are couple things to get you started.

First off, you need to be very clear in your own mind about your venture vs. your consulting work.

Consulting is simple. There are three main reasons to form an LLC as a consultant: you can isolate yourself from liability; you can snag more work as some companies prefer to work corp-to-corp rather than with individuals; and finally, you can utilize various legal mechanisms (such as managing vs. passive partners for example) to minimize your tax liability.

It sounds like none of the above applies to you very much. It's too much hassle to sue someone in a foreign country, companies prefer corp-to-corp for reasons largely irrelevant to outsourced consultants (who for example cannot claim US unemployment benefits), there is not much legal maneuvering you can do with a one-person partnership, etc.

Your venture is a whole other beast. First, keep in mind that you need to foreign-qualify your LLC in every state where you do business. It's hard to know whether you will need to foreign-qualify since you did not provide much information as to the nature of your venture, but for example accepting orders in New York means you do business there and to foreign-qualify in NY will run you over $1,000 (and if you are transacting 100% virtually, I have a hard time understanding why you need a US-based LLC). Second, if you plan to raise capital or incentivize your talent via equity, it is much easier to do so with a corporation than with an LLC. Third, there are tax implications pertaining to drawing partner distributions vs. shareholder equity, etc.

As you can see, there are no quick answers and you need to provide much more detail about your plans. In any event, I really recommend that you get a preliminary 1-hour consult with a lawyer and an accountant. If you feel that you don't need to go that far at this stage, then you probably don't need an LLC either.

To your other questions, LLC paperwork could not be easier. Book keeping is easy too, you can use quickbooks online ($17/month) or clearcheckbook.com (donation). Have an accountant show you how to reconcile your bank account once per month, and you are all set.

<comment><AUTHOR> Thanks for all the answers!

My venture would be pay-as-you-go SaaS app. But I dont think I'd be profitable right away so I'll probably keep freelancing for some time. And I sometimes get paid via ScriptLance/eLance/etc, which are US-based..

Yes, the main reasons why I want US-based LLC is that it looks much more reliable for some americans. Also, it's very expensive to form a company in my home country (Lithuania).

Now the only very unclear thing is if renting servers in US and having customers with US billing address counts as US-based incomes.

Customers probably doesnt count as 'infrastructure'. But what about servers, payment processor etc?

Does anybody know a good (and not that expensive) lawyer to advice on all that stuff?

<comment><USER2> I would say that if your plan is to provide SaaS, don't worry about it until you hit profitability. At that point, if you must have a US entity, I recommend looking into Vermont virtual corporations/LLCs.

For consulting, you could look into a Nevada LLC (cheap and low paperwork requirements) but it's not necessary. Companies that are open to outsourcing to Lithuania already know they are not working with a US-based company, while companies that want a US-based LLC will most likely have a whole bunch of "due diligence" requirements that you can't meet anyway (such as number of years in business, number of employees, etc). Trust me, your big problem is going to be getting paid as promised, not your lack of a US entity.

</comment></comment></comment><comment><USER3> _If I hire myself as a contractor in my home country, do I have to pay taxes on these incomes to US gov?

If I get paid from US-based company personally (not through company), do I have to pay taxes to US gov on these incomes?_

You don't pay taxes to US gov unless any of the following two things are true: (1) you are a US resident (2) your income is produced in the US.

There is a precise definition of being a US resident for tax purposes but it basically boils down to you staying in the US for the majority of the time.

The production of income is a bit more tricky. It basically says that if you use US "infrastructure" to produce your income then you have to pay taxes. The common example of this situation would be rental income from a property in the US. An example when this does not apply would be sitting in front of a computer in another country and writing a web app.

<comment><USER4> _rental income from a property in the US_

So if I'm hosting my app with a US hosting company, then does it apply?

<comment><USER3> No, I don't think so. The income is not produced by merely hosting the app in the US. Instead, you most likely develop and maintain it, provide customer support, etc., all outside of the US.

</comment></comment></comment><comment><USER5> _If I hire myself as a contractor in my home country, do I have to pay taxes on these incomes to US gov?_

For personal income, your country will typically have a reciprocal agreement with the US so that you don't get double-taxed. Check with your local tax body for the exact details.

(Sorry, I know nothing about the other questions.)

</comment><comment><USER6> I wonder why do you need US based LLC?

If your business is web based you can run it from anywhere... :)

Just curious...

<comment><USER7> Perhaps the poster is looking to eventually move to the US with an L-1 visa (company transfer). I've certainly considered it.

It's one of only a few ways to make it into the US on a semi-permanent basis as a self employed person unless you're ridiculously famous or rich.

</comment></comment><comment><USER8> Glad to see the tax income questions have been answered. I would like to know more about the others though. Especially :

>Is it hard to run a US-based LLC from abroad in general? Paperwork etc.

Anyone?

<comment><USER9> All Organizational forms in the US are state-level formations. That means this question is sort of difficult to answer. Some states are likely more friendly than others.

The process for citizens is generally:

\- Form Organization At State Level (Secretary Of State)

\- Apply For Federal EIN (Tax ID Number, IRS)

\- Apply For Local Tax ID Number (State Dept of Revenue)

\- File For Other Requirements -- such as local/state taxes/certificates/offices. Usually there is an employment agency you have to file with for Workers Comp.

Most states are going to need some sort of local address to form a domestic corporation/organization within their realm. You can register for business as a foreign corporation, but then you will need a local office. Again, all of this knowledge is highly state-specific, so YMMV.

Besides, you could always set it up with a UPS Store Mailbox that fwd's mail (at your cost) to your real international location.

I would also question why you need this. Do you have a specific purpose? I'm not saying you dont, I'm just curious.

<comment><USER8> Yes, I'm starting (yet another) SaaS one man startup. I need a US LLC for:

* credibility - people (especially US citizens) trust you more. Why? My personal theory - they know they have someone to sue in the US if things go wrong.

* billing - most payment processors (authorize.net comes to mind) only work with US corporations. Bummer.

</comment></comment></comment><comment><USER10> Which country are you based in? Its very important to know because local goverment might creat problem for you.</comment>


Tmux 101 - <AUTHOR> https://www.ostechnix.com/tmux-command-examples-to-manage-multiple-terminal-sessions/ <comment><USER2> Nice article, although a minor peeve is that tmux should no more have a capital letter than ls or cp.

But it goes little further than the dozens of similar articles already available. I'd like to see a few tmux 102 or 201 or whatever articles that cover some of the more advanced features such as hooks, targets and IDs, monitoring and alerts, formats and #(), command sequences, if-shell and run- shell, new/neww/splitw -P and shell scripting with tmux, buffers, capture-pane and pipe-pane, find-window and filters etc in the choose modes, pane titles, terminal-overrides, linking windows and session groups, detach -P, remain-on- exit and exit-unattached and detach-on-destroy, the mouse and advanced key bindings, the new status-format stuff in 2.9, and even menus coming in 3.0. And probably more :-).

Of course I know I could write one myself I suppose :-).</comment>


We hacked Siri on iOS 7 to interface with Spotify, Venmo, other third-party apps - <AUTHOR> http://thefourloop.com/2014/03/02/we-hacked-siri-to-interface-with-spotify-venmo-and-other-third-party-apps/

<comment><USER2> The surprising part of this is that HTTPS isn't used for the requests to Siri/Google so that this is even possible without warnings / errors.

</comment><comment><USER3> This is seriously amazing. What commands are currently available to interface with Spotify and Venmo?

<comment><USER4> There is a helper iOS app that allows for integration with Spotify/Instagram/Philips Hue, but that's not publicly available yet. Currently, it's essentially a proof of concept and you can say commands like "GoogolPlex tell me a programming joke."

<comment><USER3> Darn. Any way you'd be willing to share said helper app? If so, DM me on Twitter - @zachfeldman

I'd be happy to contribute/test stuff!

<comment><USER4> We're going to work on creating an open API so other developers can integrate their apps with GoogolPlex. So we'll definitely be in touch!</comment></comment></comment></comment>


A Billion Messages for Good: Twilio.org Launches for Nonprofits - <AUTHOR> https://www.twilio.com/blog/2013/09/a-billion-messages-for-good-twilio-org-launches-for-nonprofits.html

<comment><USER2> We're really excited to see what hackers can do to improve the world... Let's do this!

<comment><USER3> I am working on a front-end system for non-profits that will use there Twilio account information, runcampaigns.com. A discounted cost to support 501c(3) organizations will be very good.

<comment><USER4> Please tell me you're aiming for Convio and will have a reasonably reliable system...

<comment><USER3> Eventually but one step at a time. I ran a groupware platform for 9 years that was very reliable. This is my reboot of that but it needs to make money this time around so I am focusing on managing data and communication that leads to better fundraising results.

</comment></comment></comment></comment><comment><USER5> An idea (that might already exist): SMS reminders to take prescriptions. I know, for example, that my great-aunt struggles with this at 79: she's not debilitated or anything, but she's old and doesn't have the greatest memory, and remembering that she has one prescription to take twice every six hours, one to take every four hours, and one every 12 hours makes for a difficult life. (Hell, that's hard for me to remember.)

<comment><USER6> I work at Grameen Foundation, and we do that in developing countries. The system we are developing, called MOTECH, is a modular application that can be used for Pill Reminders, tracking appointments or sending periodic educational messages.

We have a lot of work still to do, both on our documentation and our code. You can read about the system here [http://www.motechsuite.org/](http://www.motechsuite.org/) or watch a video describing some of our implementations [https://www.youtube.com/watch?v=EzYy6NETvGI](https://www.youtube.com/watch?v=EzYy6NETvGI) (there are shorter videos in our channel if you don't have 14 minutes). Our code is on google code if you want to check it out or even help out [http://code.google.com/p/motech/](http://code.google.com/p/motech/)</comment></comment>


Cure for the Common Webfont - Alternatives to Arial (and Helvetica) - <AUTHOR> http://typedia.com/blog/post/cure-for-the-common-webfont-part-1-alternatives-to-arial-and-helvetica/ <comment><USER2> Subtext: these only work if you're willing to subscribe to a webfont-DRM service like Typekit. We do, and we like it, but it's a stretch to suggest that things like FF Dagny and FF DIN are immediate alternatives to Arial today; they're not, without a fair amount of black magic.</comment>


Ask HN: Can you shed some light on calculating growth? - hn-growth I'm looking for some help with growth rates. I'm trying to calculate the monthly growth rate for my app. Over the last year we went from 300,000 to 1,200,000 total user sessions. However, on a monthly basis, we had approximately 75,000 sessions completed, and it stayed consistent throughout the year. While on one hand we've quadrupled our total usage, on the other hand our growth has remained flat each month. The key question here is this - when we talk about  monthly growth, are we normally talking about cumulative usage or monthly increase in usage?
Based on the "rule of 72" it seems that quadrupling over the course of a year corresponds with a 12% monthly growth rate.
source: http://daslee.me/monthly-growth-and-the-rule-of-72
On the other hand, each month our usage stayed flat - 75,000 sessions were completed each month. From one perspective, we had 0% growth, as each month our numbers were the same. However, from the perspective of one year, we quadrupled the total number of sessions.
Is our monthly growth 0% or 12%?
Any feedback on this would be helpful. Thanks! <comment><AUTHOR> Sorry, but cumulative metrics (which is what total user sessions are) aren't very useful. To quote from [http://a16z.com/2015/08/21/16-metrics/](http://a16z.com/2015/08/21/16-metrics/) : "Cumulative charts by definition always go up and to the right for any business that is showing any kind of activity. But they are not a valid measure of growth — they can go up-and-to-the-right even when a business is shrinking. Thus, the metric is not a useful indicator of a company's health."

The more useful metrics are over a time period - like the sessions per month one. Looking at percentage growth over time for a cumulative metric seems pretty meaningless actually - among other reasons, it can never be negative even if your business just tanked.

Monthly Growth usually refers to the growth in a monthly metric per month, and would be 0% in your case.

</comment><comment><USER2> "Over the last year we went from 300,000 to 1,200,000 total user sessions."

Over what period of time? (1) 300k sessions in the previous year to 1.2M sessions this year (2) 300k TOTAL SESSIONS SINCE THE BEGINNINGS OF TIME (TSSBOF) in Jan 1st last year to 1.2M TSSBOF in Dec 31st last year

If the answer is (2), you should have seen and average of (1.2M - 0.3M)/12 = 75k sessions per month. If you saw that number every month, then essentially growth in sessions per month is zero.

The means that looking forward, unless something changes, your app will continue to have 900k sessions per year.

You just need to be clear on what you're measuring and what matters the most? The TSSBOF? Or the total sessions per month? I'd go with sessions/month.

</comment><comment><USER3> Based on my (limited) understanding, when one says "monthly growth rate", it means the delta between months. Hence, it is NOT the cumulative version.

It should be easy to see why--if you take the cumulative version, then by definition growth delta will always be 0 or positive. Even if you shut down your app next month, growth would be "0", which doesn't make sense, since your app is closed and you turned away 100% of your users so the delta in that (extreme) case would be -100% and not "0". "0" growth means there was no change in users between month.

</comment><comment><USER4> Agree you got to look @ monthly.

Now if you show some growth and want to calculate the growth rate, CMGR (compound monthly growth rate) is your best friend!

[http://www.investopedia.com/terms/c/cagr.asp](http://www.investopedia.com/terms/c/cagr.asp)

Do this: 1) X = # of sessions in october 2014 2) Y = # of sessions in october 2015 3) CMGR (compound MONTHLY growth rate) = (Y / X) ^ (1/12)-1

Here's an example: X = 100 Y = 200 CMGR = (200/100) ^ (1/12) - 1 = 5.9% monthly growth Double check...100 * (1.059)^12 = 199...damn rounding :)

Hope that helps!

</comment><comment><USER5> Think in terms of a car:

Growth metrics usually measure velocity, not speed. They don't measure the MPH that you're traveling, they measure how fast you accelerate from one speed to another.

So in your example, the speed you're traveling (75k sessions per month) is staying constant (you're not accelerating), so your velocity (growth rate), month over month, is zero.

</comment><comment><USER6> If I am reading you correctly, you are not growing. Usage is flat. So: 0%</comment>


Ask HN: Will I really lose my trademark if I don't use it as an adjective? - <AUTHOR> Bad flashbacks to grade school grammar class: today my lawyer informed me that we risk losing our trademark if we use it as a noun rather than as an adjective on our marketing site.
For those familiar with this issue, your thoughts would be greatly appreciated.
A bit of backstory: we're a tiny, two-person bootstrapped startup about to launch a SaaS product.  We've done everything ourselves, except a couple things that seemed legally intimidating and we got a lawyer's assistance with: (1) the ToS/PP for our site, and (2) a trademark application for our product/service.
We recently had a soft launch of our product, and I sent our site's URL to the lawyer so he could prepare a Statement of Use (a required submission to the USPTO to demonstrate that we are using the mark in commerce).
Well, I got a call-back from the lawyer saying that, unfortunately, there was nothing on our website he could submit for the Statement of Use, because nowhere on the site do we use the trademark as an adjective.
I was really puzzled by this — Google calls itself "Google" on its website, not "Google search engine."  "Spanx" calls itself "Spanx," not "Spanx absurdly restrictive undergarments."  Why don't they have to awkwardly adjecti-fy their marks?
He said legal precedent has shown companies who fail to use their marks as adjectives (aspirin, for example) risk losing the rights to their marks altogether.
Yet, I feel as though I rarely see this kind of adjective-style usage on tech sites.  And frankly, it would be extremely clunky to try to implement it throughout our own site.
Any wise, informed, practical, helpful, empirical, amusing, or elucidating advice for this adjective-fatigued co-founder?
Many thanks! <comment><USER2> > He said legal precedent has shown companies who fail to use their marks as > adjectives (aspirin, for example) risk losing the rights to their marks > altogether.

I am not a lawyer, but I find this odd as a generic statement and particularly with this example. It seems to conflate issues relating to genericization analysis with issues related to meeting the filing requirements and whiles it is true that use of the trademark as certain parts of speech can be relevant to genericization analysis, typically the concern is about use as a verb, or in a pluralized or possessive form if that isn't thow the trademark itself is normally. Either noun or adjective use is fairly normal—indeed, trademarks often name a product and would appear mainly as a noun.

Use as a _generic_ noun rather than a specific one for the product is the problem.

That being said, I would think adjective use inan appropriate context is an easy way to clearly meet the required demonstration of use in the specific market, though lots of the specimens submitted in trademark registrations don't show adjective use or even use in a sentence.

<comment><AUTHOR> Good point re: conflation... I'll attempt to elaborate below:

Due to particulars of our space and product name, we aren't acutely concerned with it being generified down the road.

But that aside, the lawyer suggested that not having an adjective use case to submit would hurt our chances of having our demonstration of use cleared and accepted -- period, at all, from the get-go -- which IS very concerning.

However, I had him look up the demonstration of use submitted by another company in our space, and they didn't have any adjective use cases in their (approved) submission. So I'm thinking maaaaybe this isn't as much of a concern as he's making it out to be.

It would be very awkward to put adjective use cases on our site. Plus, he made it seem as though we'd have to use it consistently, rather than just bury it somewhere.

It would be as if Adobe's page for Photoshop exclusively referred to it as "Adobe® Photoshop® software" (as dictated in Adobe's trademark guidelines) rather than just "Photoshop." That would be pretty bizarre. And after working so hard on the copy for our site, I really cringe at the thought.

</comment></comment><comment><USER3> I am not a lawyer, take this advice with a large grain of salt.

From what I understand: Your lawyer is technically correct, but possibly being overly conservative/risk-averse about it, which is the job of a good lawyer. If you trust your working relationship with your lawyer, trust your lawyers advice on this and find a way to meet your lawyer somewhere in the middle.

First of all, ordinary trademarks are no longer registered (the TM symbol is kind of unregulated at this point and only really as effective as you can prove in a court case that you've used it longer and it isn't just a generic word; you arguably don't even need to use the TM symbol itself these days). I'd say that I see a lot more TM than (R) trademarks most days, but on writing that am actually not all that certain.

If you are registering a trademark, you are registering the (R) symbol variant remains stricter about what may be registered and how. (In exchange for being harder to register and stricter in its registration you get stronger protections in court.)

If you look at examples of continued (R) usage you will find a few of the particularly older, more conservative brands do go to great pains to avoid even being used as a adjectives (they don't want to be used as real words at all) and are often "X(R) brand product" like "Kleenex(R)Brand Tissues". (Kleenex is one that is particularly conservative on this front as a lot of people do use Kleenex as a generic noun and they have been in danger of losing their trademark over the years because of that. [http://kleenex.com](http://kleenex.com) is littered with "Brand Product" speak.)

You could probably get away with not registering your trademark at all, as most startups seem to do. If you do want to register, you should probably follow your lawyer's advice, even if it does feel particularly out of date or overly conservative/risk-averse.

Hope that helps.

<comment><AUTHOR> Thanks for your thoughts on this!

We've already registered the mark -- the application was approved by the USPTO, but there's just one last step of proving that we're actively using the mark in commerce (since otherwise it's "use it or lose it").

You make a great point about risk-averse advice being good lawyerly advice. I guess where that starts to chafe a bit for me is when the risk-averse advice also runs counter to the industry standard.

It seems odd to defy industry standards in a detrimental way in the name of safety. Shouldn't following industry standards be safe enough? As noted in a prior comment, I don't see Adobe exclusively saying "Adobe® Photoshop® software" in the copy on their marketing site... they just say "Photoshop." So why can't we do that? It would be extremely awkward to do it the other way.

You may be right that we could be stuck doing it the outdated/awkward way if we want the (R), but I'm hoping that is not the case! Somehow other (R) owners have avoided this problem... fingers crossed that we can as well. It was just such odd and surprising advice to receive from him today, that I was wondering if any other startups had waded these waters.

<comment><USER3> It's interesting that you bring that up as Adobe is another one fighting to keep their trademark versus an increasing generic usage of it. Their own press kit requires that they do require the press to exclusively write "Adobe(R) Photoshop(R) software":

[http://www.adobe.com/aboutadobe/pressroom/pressmaterials/pdf...](http://www.adobe.com/aboutadobe/pressroom/pressmaterials/pdfs/photoshop_guidelines_pr.pdf)

I don't know why their own website is more lax on the subject and not held to same standards (as opposed to Kleenex's site, for example, which is). I suppose that is a question for Adobe's lawyers. (I further suppose that is an ongoing battle between Adobe's marketing department and its lawyers and probably the lawyers are shaking their heads in disgust sometimes at their own marketing website.)

«Shouldn't following industry standards be safe enough?»

Maybe? We seem to be a pendulum swing where marketing copy is more relaxed/conversational and drops TM and (R) marks and is more lax with their restrictions. This seems to me probably because there hasn't been a very high profile trademark dispute in recent internet memory, so the risk seems low and the trade-off of being more friendly/approachable/conversational in marketing copy seems worth the risk.

That pendulum swings with your competition. You need to know your competition and assuage the risk that they might want to genericize your trademark for their own competitive advantage. How much you need to do to protect your marks is probably some complicated formula of what you expect from your competition, what it is you are actually protecting, and how much effort you think may be needed to defend it in a court of law. Particularly to that last part, you likely want to defer to the risk aversion of your lawyer, presuming that lawyer is going to be the one helping you defend the trademarks in the court.

Adobe (and Microsoft and some of the other "conversational marketing" trademarks) have a fair number of lawyers on staff and a "war chest" of one sort or another in assets they can deploy to defend their trademarks. As a startup you likely don't have anywhere near those resources, so having a long conversation with your lawyer about what marks you may need to defend, against whom, and deeply assess the risk. There may be a compromise position where a conversational marketing site is accompanied by a more formal press/PR site, which may be more than enough proof that you are using the mark in commerce.

There's also a case that your competition doesn't care about your trademarks at all and there isn't any risk and you don't need to play it safe (or even have registered trademarks at all). You are in the best position to determine your risk and your lawyer is your best professional asset to work with on figuring that out.

If you think that your lawyer is being too risk-adverse you might seek other lawyers' opinions, especially a specialist in trademark disputes, but be prepared to pay for that. In that case you probably also need to be prepared to pay for that specialist's time in the case of a defense of your marks, too.</comment></comment></comment>


Do people lose interest in programming as they age - <AUTHOR> http://www.quora.com/Do-people-lose-interest-in-programming-as-they-age/answer/Kent-Beck?srid=pRa

<comment><AUTHOR> To me, programming is a way of bringing your ideas into reality. At first, it is fun to do it all yourself.

After awhile, I've discovered that I have more ideas than I can physically bring into reality myself. Then, the time sunk into programming, although fun, can be the bottleneck. So I understand eventually delegating the programming to others and having more time to directly explore and see my ideas come into life through managing others.

When I was younger I never thought this would be the case.</comment>


Twitter broadly bans any Covid-19 tweets that could help the virus spread - <AUTHOR> https://techcrunch.com/2020/03/18/twitter-coronavirus-covid-19-misinformation-policy/ <comment><USER2> Incredible. Twitter has been functioning as the self correcting mechanism throughout this crisis, as the experts have fumbled and tripped over their own feet while responding to this crisis.

As of 3 days ago, weren't the "experts" in the UK saying they were going to let the virus rip through the population to reach herd immunity? Until some dissenting experts noted that doing so would likely cause mass death and chaos. Would Twitter have shut down the members of the public noticing that the UK's plan was insane, before the change of course?

You can sure as hell bet they wouldn't shut down the UK government's initial tweets, despite the fact that they caused the virus to spread.

<comment><USER3> I remember back in the day when Twitter were free speech absolutists. And then they made exceptions just for direct threats, and just for hate speech, and just for nazis, and...

And how here we are straight-up banning speech that is contra "expert guidance".

The slopes, they are slippery!

<comment><USER4> how the hell is twitter going to know what counts as "expert guidance"? coronavirus has existed for 2 months. there are many more unknowns than facts. heres a fact: published science says closing borders to travllers is ineffective. but its done anyway (so politicians can be seen as "doing something" instead of nothing). now twitter can say they are "doing something" to stop the virus too! LOL

</comment></comment></comment><comment><USER5> > _The new policy bans tweets denying expert guidance on the virus, > encouraging "fake or ineffective treatments, preventions and diagnostic > techniques" as well as tweets that mislead users by pretending to be from > health authorities or experts._

What happens when two experts disagree with each other? This rule seems to assume that experts are all in agreement on everything.

<comment><USER6> "Preliminary investigations conducted by the Chinese authorities have found no clear evidence of human-to-human transmission of the novel #coronavirus (2019-nCoV) identified in #Wuhan, #China" \- WHO

[https://twitter.com/WHO/status/1217043229427761152](https://twitter.com/WHO/status/1217043229427761152)

<comment><USER7> I am not an expert at epidemiology but I found WHO's leadership sorely lacking, profoundly confused and generally weak in every way.

</comment></comment></comment><comment><USER8> So if I were to tweet that people should wear masks as it helps (evidence by Japan, Taiwan, Singapore, China) will I get banned as CDC says masks don't work?

<comment><USER9> Not to nitpick but CDC doesn't say they don't help, they say not to wear them if you're not sick or not caring for someone that is.

Then again CDC says a bandanna is also appropriate alternative...

<comment><USER10> Here is the thing... if you think a mask will help you from touching your face, or remind you to be cautious, wear it. It literally can not hurt.

<comment><USER11> > It literally can not hurt.

If it barely helps and hospitals everywhere are out of masks, then using one for very little gain does hurt. I think that's where we are.

<comment><USER12> I have the impression that _" it barely helps"_ is a gross exaggeration, if not a white lie, motivated by _" hospitals everywhere are out of masks"_

<comment><USER13> No.

It barely helps is a simple fact. Look at most of the masks being worn. The have huge gaps all around them. They do no cover the eyes. And if you're not used to wearing a mask you'll be adjusting it constantly -- touching your face.

What untrained people are doing is tying a fomite to their face and expecting it to magically protect them, ignoring the fact that most transmission is via your hands.

</comment></comment></comment></comment></comment></comment><comment><USER14> I'm pretty worried about this. The measures in place today are entirely warranted, and people will die if their effectiveness is undermined. But if the government announces some unacceptable measure, say a year-long shutdown for the sake of argument, tweets saying "yo wtf we're not doing that" fall under both the letter and spirit of the policy. Can we trust Twitter to revoke the policy in such a case - would they even be allowed to?

<comment><USER15> I'm not sure I follow the slippery slope. In the scenario where a tyrannical government six months from now is forcing Twitter to censor tweets... wouldn't the same tyrannical government six months from now force Twitter to censor tweets even if they weren't removing conspiracy tweets today?

Normally with a slippery slope argument there is some reason why the person who wants the final step to be taken has to take the earlier steps first, but here it just seems like the proposed bad guy can do what they want when they want to.

<comment><USER14> I don't think the government's going to force them to censor tweets as part of a tyrannical plan to suppress dissent. Twitter's just wedged themselves in a spot where they may _have_ to censor tweets until the pandemic is over, since any attempt to walk back the policy will be seen - not inaccurately! - as a decision to allow fake COVID-19 news. Maybe inconsistent enforcement of the policy will be a good enough safety valve, but I'd prefer if people judging which tweets they believe were the safety valve.

</comment></comment></comment><comment><USER16> Seems to me like the virus will spread with or without any tweets. Twitter would like to think it has enough influence to affect it in any way, but that's simply not the case. Only 10% of Americans read Twitter at all, and the percentage who would read a specific tweet is staggeringly low.

<comment><USER15> One thing we've all learned about epidemiology over the last few weeks is that like many network processes, it's not about the raw number of infected, it's about their centrality to other networks of connections. Hence you get super- spreaders, like how over 80% of the cases in South Korea are directly traced to one person going to church.

If we imagine that news and reporting works the same way, then what matters is not that only 10% of Americans read Twitter, but which 10%. If, say, all journalists and reporters and members of congress read Twitter, then that would be a conduit for misinformation posted on Twitter to spread more broadly, right?

(I do agree that in the past Twitter's role in civic life has been overblown, e.g. westerners projecting Twitter as being operative in the Arab Spring when in fact most domestic organization in the Arab Spring was done over local mesh network apps and Twitter was mostly used to report the events to the English- language world, not initiate change internally)

<comment><USER17> Careful. Remember, it only counts as a case if it's detected, and South Korea- style contact tracing is a lot less capable of detecting small clusters that have split off from each other due to the people tying them together no longer testing positive than they are large clusters involving the same people going to the same church or office with attendance-tracking on a regular basis.

</comment></comment></comment><comment><USER18> I think Twitter's motivation isn't to reduce the impact of the virus, or that their management trusts governments this much. They're anticipating politicians looking for scapegoats, they're ensuring they can't take the blame from governments following any negative outcomes.

From Twitter's standpoint, if as a business their primary goal is to continue to at bare minimum exist, this is actually their most conservative and lowest risk option. We could debate all day on what sources of information have been the best, government or non-government, but I don't think Twitter is interested at all in considering that side of it.

</comment><comment><USER19> Any censorship cause a rather strange expression to overcome it. Example: YouTube automatically disable the monetization on video which utter the words coronavirus. The intent is to prevent malicious videos profiting from spreading the bogus information like drinking a certain herb tea makes you immune to the virus.

The YouTube's intent is understandable, but it ban without considering the context, just blind matching to the word, even if the context is harmless.

The result, the youtubers start referring the coronavirus in an expression "the virus we can't say its name here" and alike.

It reminds me of the Harry Potter's You-Know-Who.

</comment><comment><USER20> It's unfortunate that the success of twitter relies on .. going viral.

</comment><comment><USER21> The thing I am worried about is excessive power to shape public discourse.

I am certain the power to decide what is and what is not acceptable should not be vested in private companies.

</comment><comment><USER22> twitter is so bad at policing anything, just be a platform, and stop shaping the conversation

</comment><comment><USER23> cool now do that to the white supremacist side of twitter

<comment><USER24> It pains me to think that this will probably never happen.

<comment><USER25> Must be really painful to know your political opponents are not being oppressed. White supremacists probably even feel the same way about you!

<comment><USER24> White supremacy is more than mere "political opposition." Don't trivialize that.

</comment></comment></comment></comment><comment><USER26> The virus doesn't care about our posts. It does not read them, does not try to influence them. It is mindless and it may not even be alive, depending on the definition of life.

The tools you use to counter a disinformation campaign are not the tools you use to stop a virus. And with authorities actually getting stuff wrong stopping heterodox views is a suicide pact.

China is doing this because they're worried about the stability of their political system. Ours will survive goldbug1989 posting about how much they hate the quarantine or karenlovesyoga writing that essential oils will protect us.

</comment><comment><USER27> That's really awful. Twitter has become very heavy-handed lately. Even accounts that just tweets about American politics have been suspended. For example, there's some videos of a certain politician touching women and children inappropriately and those who have tweeted or retweeted some of those videos have either been banned or have had their access restricted.

Yes, I know Twitter is a privately-owned site and they can do whatever they want with their site. But it also means that it should no longer be seen as a neutral platform.

I really wish some hackers got together and created a censorship resistant social media platform. It is sorely needed.

<comment><USER28> "But it also means that it should no longer be seen as a neutral platform."

Bizarre to me that anyone would ever have thought that in the first place. Twitter has always been a for-profit enterprise, which completely precludes neutrality.

"I really wish some hackers got together and created a censorship resistant social media platform. It is sorely needed."

And then what happens when that platform is used to plan and execute violent attacks and share illegal pornography? I don't disagree that this is a good idea, but it's also not a silver bullet. Free and open communication will always be a perpetual struggle requiring trade-offs and compromises.

<comment><USER27> If that is bizarre to you then you are simply out of touch. A majority of Twitter users treat the site as neutral carrier of tweets. Nothing happens to a censorship resistant platform that is used to execute violent attacks or share illegal pornography. And that is the beauty of it. Your question is like asking what happens to the room in which I and my co-conspirators plan a coup.</comment></comment></comment>


A Mathematical Guide to the Best (And Worst) Ways to Lace Your Shoes - <AUTHOR> https://www.amazon.com/Shoelace-Book-Mathematical-Guide-Worst/dp/0821839330 <comment><USER2> I suspect Ian's Shoelace Site[1] probably has this covered.

(I use Train Track Lacing on my interview shoes. Looks good and evens out the pressure.)

[1] [https://www.fieggen.com/shoelace/index.htm](https://www.fieggen.com/shoelace/index.htm)</comment>


All Facebook posts now searchable; or: All your posts are belong to Us  - <AUTHOR> http://arstechnica.com/business/2013/09/all-public-facebook-posts-ever-made-are-now-searchable/

<comment><AUTHOR> As someone with a deleted account I hope this doesn't mean that my now non- existent privacy settings open me up to search.</comment>


Warning over narcissistic pupils - <AUTHOR> http://news.bbc.co.uk/2/hi/uk_news/education/7943906.stm <comment><USER2> As a product of the overabundant self-esteem training myself, I can verify that my narcissistic tendencies (coupled with my deep-rooted insecurities) has significantly (negatively) affected my relationships (of all types).

<comment><USER3> same here. fortunately, it can be un-trained. or rather, better habits can be trained. there is no end, just little steps along the way.

</comment></comment><comment><USER4> How many of you start-up founders would like to hire a narcissist convinced he can do no wrong?

"Narcissists make terrible relationship partners, parents and employees. It's not a positive characteristic. We are in danger of encouraging this," she said.

"And we are kidding ourselves if we think that we aren't going to undermine learning if we restrict criticism.

"Parents no longer want to hear if their children have done anything wrong. This is the downside of the self-esteem agenda.

"I'm not saying it's of no value--but you get unintentional consequences."

</comment><comment><USER5> One of the biggest problems is teachers giving non-constructive criticism to students.

Back in elementary school, I remember a teacher telling me how bad I was in history, but in only a harsh manner. What happened to teachers being able to "make friends and influence people" while still being able to get a point across?

<comment> puzzle-out Too true. Then again, should teachers try to be the students friend? When I was teaching I saw way too many young teachers more concerned with being popular, whereas actually the kids seemed to respect the teachers that kept a firm dividing line. There's something very sad about an adult trying to get social assurance from children.

</comment></comment><comment><USER6> More nuanced, video featuring Dr Carol Craig

[http://www.ltscotland.org.uk/video/c/confidenceandselfesteem...](http://www.ltscotland.org.uk/video/c/confidenceandselfesteemcarolcraig.asp?strReferringChannel=learningaboutlearning)

</comment><comment><USER3> haha. i was wondering how narcissism and eye balls related.</comment>


How to Get a VC Meeting – the flowchart - <AUTHOR> http://steveblank.com/2012/10/17/how-to-get-a-vc-meeting-the-flowchart/

<comment><USER2> The flowchart is both correct and completely useless. It betrays the concern of a successful entrepreneur (Steve Blank) and focuses on the status of the VC with whom you will meet (partner, associate or none).

VC meetings are easy to get. Just go to a few networking events, meet some of them, get their business cards and drop them a note asking for a meeting. Their job is something like 40% networking, 40% saying "no" to deals and 20% trying to value deals to which they might or have said "yes". By and large, I've found them to be very smart, nice and helpful people and I count a few of them among my friends.

But remember: to get advice, ask for money; to get money, ask for advice. VCs apply money to fix problems and risks in order to increase growth. Help them understand how you can help them through helping their reputation, portfolio (through helping your business) and future opportunities (through helping you).

<comment><USER3> A slight deviation of topic, but I think this point is too-often ignored: "VCs apply money to fix problems and risks in order to increase growth." VCs want you to increase your risk profile, while entrepreneurs focus on getting capital to decrease the risk of going belly up from undercapitalization. Your last point seems like the way to avoid those two from coming in conflict from the outset.

</comment></comment><comment><USER4> Just a counterpoint...

Before even trying to get a VC meeting I'd ask myself, "Can I do this without a VC?" All money comes with strings attached, VC money more than customer money.

There's no one correct answer but if you really believe that your web startup can succeed I'd consider first try filling out your team by adding partners with complementary skills and then put the same effort into sales and marketing to the customer rather than VC's. VC's could be harder to close than typical customers.

What's the main motivation for the VC route? Shooting for the moon? Fear of working without a salary? (nobody's going to pay you while you try to raise money either)

</comment><comment><USER5> My understanding is that VC's, partners or associates, whole job is to source deals. It's their job to take meetings. What happens after that meeting is the tricky part.

</comment><comment><USER6> These days everyone is a funding expert for start-ups. Trying to earn money on clueless founders is I think getting a real big market. Providing services and training to these founders is apparently easy and without any risk. Is Steve Blank one of them?

</comment><comment><USER7> VC meetings aren't that hard... filtering out the VCs who are just fishing for information can be harder.</comment>


Apache Spark: The Next Big Data Thing? - <AUTHOR> http://blog.mikiobraun.de/2014/01/apache-spark.html <comment><USER2> MLlib [1] is definitely worth checking out as a Spark library for common ML algorithms - it's crazy how little code is needed to implement efficient, distributed inference algorithms.

Spark's primitives (especially the RDD abstraction) make it feel like a DSL for distributed ML algorithms - for example, check out the implementation of distributed alternating least squares matrix factorization at [2] - ~100 lines of Scala code once stripped of the Java interoperability boilerplate.

On the weekend, I took a crack at implementing some distributed ML algorithms using Alternating Direction of Multipliers (ADMM) in Spark, following [3]. In a few hours and ~400 lines of Scala code, it was possible to implement distributed versions of L^1 regularized logistic regression, ridge regression, and SVMs [4].

It's a very impressive framework, and very much empowering.

[1]: [http://spark.incubator.apache.org/docs/latest/mllib- guide.ht...](http://spark.incubator.apache.org/docs/latest/mllib-guide.html)

[2]: [https://github.com/apache/incubator- spark/blob/fdaabdc673875...](https://github.com/apache/incubator- spark/blob/fdaabdc67387524ffb84354f87985f48bd31cf60/mllib/src/main/scala/org/apache/spark/mllib/recommendation/ALS.scala)

[3]: [http://www.stanford.edu/~boyd/papers/pdf/admm_distr_stats.pd...](http://www.stanford.edu/~boyd/papers/pdf/admm_distr_stats.pdf)

[4]: [https://github.com/ajtulloch/admmlrspark/tree/master/src/mai...](https://github.com/ajtulloch/admmlrspark/tree/master/src/main/scala)

</comment><comment><USER3> I believe the answer to that question is yes. I've raved about Spark many times on HN before. We're using it and absolutely love it.

Not sure why the author thinks there's no built-in support for iterations. The support is there, native scala/java. But you might have to collect to the driver/master to check for convergence, for example. Their simple logistic regression example doesn't check for convergence, but hard codes the number of iterations. [http://spark.incubator.apache.org/examples.html](http://spark.incubator.apache.org/examples.html)

Some quirks that I've learned. When doing groupbys or other join operations, it helps to specify the number of partitions -- actually this is true in general, but more so for join-based operations, otherwise you can run into memory/gc issues. The equivalent is of course, setting the number of reducers to something which ensure you won't run out of heap space.

Because spark serializes the closures which transform your data, if you don't cache (at least via disk, using persist), then when iterating over an RDD or re-using it, you'll just waste cycles.

Beyond that, as an experienced scala developer, I've found that Spark feels incredibly natural. And being able to run things on the repl cannot be appreciated enough.

</comment><comment><USER4> Spark cleans up some abstractions found in the Hadoop ecosystem but I would hesitate to call it the "next big thing" because it doesn't really address any of the core weaknesses of Hadoop. In Big Data, there are three touchstone applications that the current generation of platforms largely do poorly: real- time, geospatial, graph. Spark does not really address any of these. It might be more accurate to say that Spark is the "next big thing" in the _Hadoop_ universe but still retains many of the same limitations on expressiveness, thereby creating opportunities for a more general Next Big Thing in Big Data.

Specifically, "faster batch + in-memory" is basically a simple patch on the batch mode problem but does not really address continuous flow real-time models. At large scales it is quite difficult to get robustly efficient behavior out of a parallel system that is batching things on 5 second intervals if the data flow is anything but trivial.

For geospatial and graph analysis, Spark appears to retain the same limitation of Hadoop in that it cannot deal with data models and operations without an _a priori_ optimal partitioning function. Static hash and range partitioning won't cut it, particularly if the streaming data sources are actually real- time. The ability to generate uniform partitioning of complex data models with inherently unpredictable data distributions is critical to parallelizing some important analysis types but there is no obvious support for such mechanisms in Spark.

One could look at Spark as Hadoop done right, more or less.

<comment><USER5> GraphX ([http://amplab.github.io/graphx/](http://amplab.github.io/graphx/)) is essentially GraphLab2 ([http://graphlab.org/](http://graphlab.org/)) built on Spark. Joey Gonzalez, the creator of GraphLab, is also the GraphX lead.

Here are the papers for GraphLab and GraphX...

GraphLab: [http://graphlab.org/home/publications/](http://graphlab.org/home/publications/)

GraphX: A Resilient Distributed Graph System on Spark ([https://amplab.cs.berkeley.edu/publication/graphx- grades/](https://amplab.cs.berkeley.edu/publication/graphx-grades/))

See also "Introduction to GraphX - Presented by Joseph Gonzalez, Reynold Xin - UC Berkeley AmpLab 2013" ([http://www.youtube.com/watch?v=mKEn9C5bRck](http://www.youtube.com/watch?v=mKEn9C5bRck))

</comment></comment><comment><USER6> Spark does look very promising. It is great being able to experiment with jobs on distributed collections from a Scala repl. Consequently it is very quick to get started.

My first impressions are that there seems to be a fair amount of abstraction leakage. Some things that compile and look valid fail at runtime - e.g. referring to other RDDs from within a filter predicate or map function. Other More complex jobs cause the nodes to fail and it gets into infinite loops of restarting the nodes, replaying the job, and them dieing again.

I hope once I get a better understanding of what is going on underneath I will understand what is going on here.

<comment><USER7>
        More complex jobs cause the nodes to fail and it gets
        into infinite loops of restarting the nodes, replaying
        the job, and them dieing again.

This is probably a bug in your code. Debugging these cluster applications does take some getting used to. You'll want to look at the stderr output of the failing executor, and you'll probably see that it's dying due to some kind of exception. You can do this by visiting port 8080 of the master node over HTTP, i.e. [http://mymaster:8080](http://mymaster:8080). Feel free to email the Spark users list if you have any questions: [https://spark.incubator.apache.org/mailing- lists.html](https://spark.incubator.apache.org/mailing-lists.html)

It's true that you do have to understand the programming model and some details of how it's implemented to use Spark effectively. However, any abstraction that was "pure" and perfectly non-leaky would necessarily sacrifice some performance and transparency to achieve that goal. Spark aims to be both high-level _and_ high-performance.

Full disclosure: I'm on the Spark team at the UC Berkeley AMPLab.

</comment></comment><comment><USER8> I would like to play with Spark, have anyone tried any of the Docker images? There's too many =/

<comment><USER9> The best way to try out Spark & related tools is to follow the 2 days exercises from the AMP Camp 3, they have a script to launch the whole cluster (Berkeley Data Analytics Stack) on ec2 very easily so you can get your feet wet without too much effort.

[1] [http://ampcamp.berkeley.edu/big-data-mini- course/](http://ampcamp.berkeley.edu/big-data-mini-course/)

[2] [http://ampcamp.berkeley.edu/big-data-mini- course/launching-a...](http://ampcamp.berkeley.edu/big-data-mini- course/launching-a-bdas-cluster-on-ec2.html)

<comment><USER8> two days? :O

<comment><USER10> I helped write the AMPCamp exercises (and parts of Spark) - the exercises were done in the context of a two-day long seminar held at Berkeley where the days were mostly filled with talks/training about the stack. The actual exercises were designed to take a couple of hours each day, with plenty of time for questions/etc. You can probably get through them in an evening, and I've seen people do it in less!

</comment></comment></comment></comment><comment><USER11> I love Spark, is about time to start the NoHadoop movement!

<comment><USER12> In the next year I can imagine most people deploying Spark will be doing it on Hadoop, since Cloudera 5 will support Spark. It's a natural fit, most people don't hate HDFS but their use case doesn't naturally fit the MR programming model.

<comment><USER11> yeah, I mostly complain about trying to fit everything to the Map/Reduce model

<comment><USER13> The "Hadoop ecosystem" is getting less and less about MapReduce, and more about other execution frameworks that can share HDFS with it.

<comment><USER12> You don't happen to work at Cloudera, do you? I noticed you have some submissions about Impala and Oracle being evil, which seems to be a pretty common view among the ex-Oracle DBAs there

<comment><USER13> I do happen to work at Cloudera (hence the Impala submissions), although I'm neither an ex-Oracle DBA nor a huge believer that they're evil. I really don't have a lot of first-hand experience with Oracle as a company - which as you'll see, is why my submission was actually a question about the community's perception and why that's a common view.

<comment><USER14> Hey, I've recently begun the interview process at Cloudera, would you mind sending me an email? My address is in my profile, and I'd love to ask you a couple questions.

</comment></comment></comment></comment></comment></comment></comment><comment><USER15> It is no coincidence that we're seeing the theme of decentralization cropping up everywhere. Consider an investment approach based on the assumption decentralization is getting ready to eat the world.</comment>


The Radial Tire Lesson for Silicon Valley - <AUTHOR> https://www.wsj.com/articles/the-radial-tire-lesson-for-silicon-valley-1496604821 <comment><AUTHOR> I bypassed the paywall by clicking on the URL in Kessler's tweet about his column: [https://twitter.com/andykessler/status/871464537911840768](https://twitter.com/andykessler/status/871464537911840768)</comment>


Networks, Crowds, and Markets: Reasoning About a Highly Connected World (2009) - <AUTHOR> http://www.cs.cornell.edu/home/kleinber/networks-book/

<comment><USER2> I've been working my way through this book (printed version, I'm currently only about 1/4 through it) as a self-study exercise. I really like the book so far. I think the exercises could be made more challenging, and it's at times a little more verbose than I'm used to, but overall I'm finding it an enjoyable experience and I'd recommend it to someone who's interested in an introductory level view of the topic.</comment>


Project Roslyn: Exposing the C# and VB compiler's code analysis - <AUTHOR> http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2012/Roslyn

<comment><USER2> I remember developing in Visual Basic 3 and being able to decompile source code using an amazing program called Dodi (German Developer). I was 13 at the time but it was a scary thought of mine that that anyone could decompile my hours of hard work to re-release the same product in a matter of seconds. I'm happy to see Microsoft still going forward with C# and VB.NET attempts to make their code more open to the web and Microsoft compatible products yet less effort on the developers part. I still use Visual Basic to develop little tools at the speed of light. Not sure too many companies that would go to industry standards but this could be very intriguing.</comment>


AT&T Will Soon Let Magic Leap Users Watch Multiple Shows at Once - <AUTHOR> https://www.adweek.com/digital/att-will-soon-let-magic-leap-users-watch-multiple-shows-at-once/amp/ <comment><USER2> Is there any news of DirecTV Now allowing multiple streams on say an Apple TV?

I've heard other apps like MLB and stuff allow multiple streams.

I'd love to watch a movie / tv show and a news network in PIP or side by side or.</comment>


Flying While Fat - <AUTHOR> http://flyingwhilefat.com/ <comment><USER2> I get this error in the video: "YouTube - Playback on Other Websites has Been Disabled by the Video Owner"</comment>


Propaganda Posters of the Chinese Space Program (1962-2003) - <AUTHOR> http://www.openculture.com/2015/01/vintage-posters-of-the-chinese-space-program-1962-2003.html

<comment><USER2> I picked up a Chinese propaganda poster calendar a few years back. All the smiling chubby babies...</comment>


Ask HN: Did we all give up on a better search engine/experience? - <AUTHOR>

Have we settled for Google? <comment><USER2> No. I think Google is still the best of its kind in many product areas, but every day I am looking for the new thing that wildly exceeds it, even if only in one area - I feel we're stuck in a local maximum right now that will look laughably restrictive later. In particular, I desperately want to find an alternative to Google for News. Google News is so laughably, self-evidently bad that I have to believe either nobody is working on it or that they enjoy pottering about in their IDE more than they like reading news. Well, it could also be that that whatever they are doing now maximizes revenue for Google even if it doesn't serve consumers very well, so that they know it's bad and don't care, but I don't want to be so pessimistic.

Seriously, I want to go and (metaphorically) kick in someone's door in Mountain view, wipe the nearest whiteboard with a startled junior engineer, and write a a new set of UI priorities for that project before security arrives to eject me from the building. It should be one of their best products, and it's a joke.

<comment><USER3> Public visibility of a Google product and the size of the team working on it often aren't correlated. It wouldn't surprise me if News is a very small team that's spending most of their time doing maintenance.

</comment></comment><comment><USER4> I often want my searches to be confined to long form content like blog articles or magazine articles, but google insists on including vaguely related shopping sites, discussion forums or company product pages in the results. They had a prominent search in blogs option in the past, but it's now available only via some URL parameter hack.

I'd really appreciate a search engine that confines itself to blogs.

</comment><comment><USER5> I gave up on "better" but I still use DuckDuckGo (because I don't like the idea of Google). The important thing is being able to type the !bang things to search a particular website (with their own search engine) but I find myself increasingly typing !g

</comment><comment><USER6> Nope, DuckDuckGo is my default search engine, make it yours too.

</comment><comment><USER7> What kind of improvements would people like to see? I haven't really thought about it much, but I feel google finds what I need.

<comment><AUTHOR> I am not sure, this is a "you don't know what you don't know case," but certainly this cannot be the best today.

</comment></comment><comment><USER8> Vertical search continues to be Googles largest threat. New kinds of AI based vertical search experiences will lead the way.

<comment><AUTHOR> What example of vertical search engine do you have in mind?

For example I do not buy it when people call Amazon search a vertical SE. They just have search for their own database.

</comment></comment><comment><USER9> Google always finds what I am looking for, which is the task of a search engine. Why is that settling?

</comment><comment><USER10> No!</comment>


NeXT logo by Paul Rand (2010) - <AUTHOR> https://www.logodesignlove.com/next-logo-paul-rand <comment><USER2> Love this

> When Jobs was asked what it was like to work with Rand, he said, "I asked > him if he would come up with a few options, and he said, 'No, I will solve > your problem for you and you will pay me. You don't have to use the > solution. If you want options go talk to other people.'"

<comment><USER3> Web design, every day. Customer has a shit site, they know it's shit, they want to pay someone $$$ to do a new site, and they want to change 75% of the new design because they think they know better than the designer...

<comment><USER4> I noticed the guy who designs icons for me always leaves what appears to be a deliberate mistake, I then point it out and it gets fixed. I know what he is doing and I'm pretty sure he knows I know what he is doing.

<comment><USER5> It's called The Duck Technique, after an apocryphal story of an animator for Battle Chess.

[http://pud.com/post/59851751577/the-duck- technique](http://pud.com/post/59851751577/the-duck-technique)

[https://en.wikipedia.org/wiki/Battle_Chess#Development](https://en.wikipedia.org/wiki/Battle_Chess#Development)

<comment><USER6> I've been tempted to make a Removable Duck for Unity3D and sell it on the Unity asset store for hourly freelance developers to use.

It will quickly pay for itself: The developer can drag a duck into the scene and configure it to define how many hours it will take to remove, and what kind of removal interface and effect to use. Then when they show the demo to the client, and the client demands "Remove the duck!", then can say "Ok. That will take X hours." Then they simply type some secret hot keys or click on some invisible hot spots for a while, and the duck will pop up a count-down timer and slowly start shrinking or dissolving or whatever, until it finally disappears after X hours with a puff of smoke, and then the developer can bill the client for that many hours.

</comment></comment></comment></comment></comment><comment><USER7> Personally I hate the NeXT logo. I've always thought about just how awful I think it looks every time I see it. This is the first time I've got the sense that people genuinely love the logo. Very interesting!

<comment><USER8> The logo looks great on an actual NeXTstation, cube, or MegaPixel, since those are entirely black. The logo pops right out. It really does look good IMHO.

<comment><USER9> It'd be interesting to imaging the modern evolution of it equivalent to what has been done to the Apple logo over the years, appearing in the same places the Apple logo does today.

<comment><USER8> Yes! I'd like to see that, too.

</comment></comment></comment></comment><comment> 52-6F-62 I never knew who Paul Rand was. This is great. Especially whoever linked the list of his work.

I wanted to draw the mild comparison to a Canadian with a similar cultural impact here: Allen Fleming.

Unfortunately Fleming died young otherwise his work might Have pervaded.

He designed the CN (Canadian National Railways) logo that's still used, Ontario Hydro, was art director at Maclean's magazine, Ontario Sciene Centre logo, and was responsible for many designs at Cooper & Beatty Type— including the type-o-file. He also literally wrote the book on designing Canadian stamps as I just found out.

[http://www.marthafleming.net/allan-fleming- project/](http://www.marthafleming.net/allan-fleming-project/)

[https://www.google.ca/amp/s/designkultur.wordpress.com/2010/...](https://www.google.ca/amp/s/designkultur.wordpress.com/2010/07/21/logos- graphic-artists-allan-fleming-tracing-the-evolution-of-the-cn-logo-50th- anniversary-1960-2010/amp/)

[http://www.eyemagazine.com/feature/article/allan-fleming- the...](http://www.eyemagazine.com/feature/article/allan-fleming-the-man-who- branded-a-nation)

[https://gdc.design/fellows/allan-fleming- fgdc](https://gdc.design/fellows/allan-fleming-fgdc)

</comment><comment><USER10> He designed the IBM logo too? Literally my two favorite logos of all time! He's a genius.

<comment><USER11> It's shockingly hard to find a single list, but yeah, he was prolific and extremely effective.

He also did the UPS, PBS, ABC, Enron, westinghouse, and surely many other big ones that slip my mind.

PBS and westinghouse are especially favorites of mine.

[https://en.wikipedia.org/wiki/Paul_Rand](https://en.wikipedia.org/wiki/Paul_Rand)

<comment><USER10> I found a single list!

[http://www.paul-rand.com/foundation/identity/](http://www.paul- rand.com/foundation/identity/)

<comment><USER12> LOL, that Enron logo was pretty sweet!

<comment><USER13> I am sure the similarity with the E Corp logo in Mr Robot is pure coincidence (or maybe not) [http://mrrobot.wikia.com/wiki/E_Corp](http://mrrobot.wikia.com/wiki/E_Corp)

<comment><USER14> It even says so in the trivia section on that very page: "During an interview on Google, Sam Esmail, the creator of the the series, stated that E Corp's logo is actually the Enron logo."

</comment></comment></comment></comment></comment></comment><comment><USER15> Near the Ottawa tech distinct in Kanata we have the NeXT Restaurant which has a very similar logo:

[http://www.nextfood.ca/](http://www.nextfood.ca/)

I guess now that NeXT has been defunct for decades, being inspired by its logo isn't a big deal.

<comment><USER16> Apple actually still maintains the NeXT trademarks[0].

[0] [https://www.apple.com/legal/intellectual- property/trademark/...](https://www.apple.com/legal/intellectual- property/trademark/nexttmlist.html)

<comment><USER17> A trademark in the technology class would not apply to a restaurant anyway. [https://en.m.wikipedia.org/wiki/International_(Nice)_Classif...](https://en.m.wikipedia.org/wiki/International_\(Nice\)_Classification_of_Goods_and_Services)

</comment></comment></comment><comment><USER6> I can't remember where I heard this story, and don't know if it's true, but the story was:

Steve Jobs commissioned some famous European designer to come up with a revolutionary design for his awesomely great NeXT computer. When they flew out to see the design, it turned out to be shaped like a human head, so a pissed off Steve Jobs and his team got back on their plane and flew back to the US.

There's some mention of it here:

[http://www.complex.com/pop-culture/2013/10/steve-jobs- most-i...](http://www.complex.com/pop-culture/2013/10/steve-jobs-most- important-power-moves-while-at-next/the-nextcube-design)

Known for his fanatical approach towards hardware design, Stevie was poised to make the company's first-ever computer a machine that embodied power and strength. Repulsed with the results of the several design houses he hired to create protoypes, including one that resembled a human head, he brought in the designer of the Snow White design language, Hartmut Esslinger, to create the final product. Jobs immediately labeled it the NeXTcube when shown the end result. Even though the machine stoodout as a complete failure, it was the NeXTcube's groundbreaking form factor that established the radical design identity of Apple's future desktop machines.

</comment><comment><USER18> If you're new to Paul Rand, there is a very good book on his work:

Paul Rand [https://www.amazon.com/dp/0714839949/ref=cm_sw_r_cp_api_4GO2...](https://www.amazon.com/dp/0714839949/ref=cm_sw_r_cp_api_4GO2AbGQ211B1)

</comment><comment><USER19> It isn't of practical value, but it makes me happy to discover another distinguished member of the Secret Council of /(\b[plurandy]+\b ?){2}/i.

</comment><comment><USER20> How To Burn $100,000 In 100 Pages, the book.

Logos are pretty easy to make, rarely important, and spending excessive money on them is always a vanity (or incompetence) expense by founders / C suite. Companies/products make the logos, not the other way.

<comment><USER21> Well, sometimes. It's no coincidence that the Nike swoosh fits perfectly along the side of a shoe. If you're a design company and you can integrate your logo as a design element and it adds something instead of standing out as a gratuitous advertising badge, you have an advantage. See also: the three Adidas stripes.

</comment></comment><comment><USER22> This guy has a pretty diverse skill-set! He's great at graphic design, debating, and politics.

</comment><comment><USER23> I hate making these types of trashy low class reddit-esque quips, but did everyone see the guest appearance by Napoleon Dynamite towards the end of the clip?

</comment><comment><USER24> (2010)

> _The NeXT logo mightn't be a classic [...]_

Excuse me!?</comment>