The Science and Art of Practicing - <AUTHOR> http://www.violinist.com/blog/laurie/20146/15906/ <comment><USER2> I found a lot of improvement in my guitar practicing by reading "Fundamentals of Piano Practice."

The author made it available for free at [http://pianopractice.org/](http://pianopractice.org/)

As I play fingerstyle I study hands separated by separating the bass and melody as thumb and fingers, then move to hands together.

I also analyze the piece to find the hardest as well as the most repetitive measures, work those out and pick up the rest of the piece. You also work these measures with transitions, by playing the preceding measure or half measure leading in and out of the measure you're practicing.

I typically don't get a lot of time to practice so I will pick up my guitar at random and play measures that I'm working on.

</comment><comment><USER3> Awesome article. But tbh still very vague, I'd love to hear about some of your guys' specific practice schedule. Below is mine,

Guitar Practice: I do repertoire, technique and improvisation. For repertoire, I usually focus on the hardest passage and run it three times to the best ability and then connect the whole song together for 3 goes before I finish. For technique, I usually rotate on scales patterns/licks/ear training; review the last pattern I learned and learn a new pattern. For improvisation, I just doodle with jam track and listen and try to iterate on how I can improve (e.g., better timing on stressing notes on chord changes, building tension).

Dota Practice: I sometimes watch youtube casts on warding and laning. Also what different heroes are good to counter different heroes. But mostly I just play ranked games and try different heroes.

<comment><USER4> I play trumpet, my routine usually starts with long tones and slow articulation exercises. As I get warmed up I extend into lip slurs, faster articulation, and flexibility. After I'm completely warmed up then I go into a mix of technique work, etudes for fun, and whatever other repertoire I need to work on for performance.

Repertoire that I'm just maintaining is a mix of full runthroughs and reinforcing small segments. For new repertoire I usually work through it measure by measure, mixing air+valves reps with playing reps to save my lips.

</comment></comment><comment><USER5> Good advice. My wife (a pianist and piano teacher) just read the article kindly submitted here and said that that is how she practices. I've heard over the years about how she advises her students, and, sure enough, one of the key ideas is to distribute practice across multiple days each week rather than piling into one lump of time just before the lesson.

After my reading of the article, I have to say that getting a good night's sleep each night is definitely crucial for learning anything challenging. Sleep makes connections and consolidates learning.

The one thing I don't see mentioned in this article that my wife tells most of her students, after learning about it from one of her university music teachers, is how important it can be to analyze a new piece of music through multiple modalities. My wife's university teacher has a book about "music mapping,"[1] an approach to reading musical scores and adding visual notations to them, which is highly individualizable and works wonderfully for helping musicians learn where the music is in a new piece of music. Playing with musicality rather than just memorizing notes is always the key idea.

[1] [https://www.areditions.com/books/rs001.html](https://www.areditions.com/books/rs001.html)

<comment><USER6> > getting a good night's sleep each night is definitely crucial for learning > anything challenging.

That makes me wonder why sleep deprivation is such a large part of the college experience.

<comment><USER7> I think pulling 'all nighters' (seriously, being in the computer lab when the sun comes up in totally not cool - especially with the gradual lightening of the sky and realization that you are bone tired and still have 2 hours left of work...) is part of the culture that college students expect at some but not all schools. Similar to binge drinking, there is a lot of peer pressure to 'pull an all nighter' or 'do a funnel (of beer)'.

Peer pressure doesn't stop after high school. In fact, it might not "stop". As some of us grow older, your peers change until they reflect what you really want in life (for most, in my experience, that means having peers who are settling down and having two kids and a family). I'm sure there is definitely peer pressure at senior citizen/assisted living homes to be 'cool' (whatever the norm is).

Startup culture is similar in the bubble of San Francisco. The norm is so skewed from the median/average of what most people in a business environment do.

As David Foster Wallace said, the fish doesn't know it is in water. [1]

[1] [http://www.wsj.com/articles/SB122178211966454607](http://www.wsj.com/articles/SB122178211966454607)

</comment></comment></comment><comment><USER8> _In one study, where they tried learning both ways, people felt like blocked practice was better, even though their ultimate performance was measurably better after the random practice!_

 _Though you might improve at a task during practice, you're less able to carry forth that improvement to the next day._

 _Studies have shown that consistently getting a full night's sleep (eight hours) plays a huge role in learning motor and auditory skills_

 _The surprising parts are usually the places that they did not account for in their mental practice._

 _Forget about the nasty passages for now. Instead, spend two weeks getting in touch with the musical aspect of the piece and practice it through, just for musicality._

The linked essay in the article also gives a timeline on how long in takes for neurons to connect: it takes about a week to happen.

[http://mollygebrian.files.wordpress.com/2013/10/what- musicia...](http://mollygebrian.files.wordpress.com/2013/10/what-musicians- can-learn-about-practicing-from-current-brain-research.pdf)

Also: _You will be much better off practicing your orchestra music for 15 minutes a day until the concert, rather than "wood-shedding" the day before the concert. Why? Because you'll have all those nights of sleep for your brain to process the new music._

</comment><comment><USER9> Good article, i discovered it right after i bought my violin (could have been after i got a viola, can't remember). I think you can apply these tips to learning, say, rust programming, or convex optimization. I also think a random beat generator is sort of similar to backing tracks i record in Garage band or ableton, and you should leave pieces/charts you're learning out on the stand next to your computers, and look at them often, 5x/hour.

Another tip, always remember the first time you tried to play a clarinet, cello or violin (if you can). Those for me were special moments.

Here's some other (comprehensive) books about practicing, which go from mechanical prescriptions about washing hands and brushing teeth beforehand, to the zen, in the vein of the motorcycle and archery books, like long tones/long bows/drone notes/son filé (the last is Galamian's term, his violin technique book highly recommended).

Kenny Werner, [http://www.amazon.com/Effortless-Mastery-Liberating- Master-M...](http://www.amazon.com/Effortless-Mastery-Liberating-Master- Musician/dp/156224003X) (recommends pianists practice long tones ?!)

Sterner: [http://www.amazon.com/The-Practicing-Mind-Developing- Discipl...](http://www.amazon.com/The-Practicing-Mind-Developing- Discipline/dp/1608680908)

Bruser: [http://artofpracticing.com/book/](http://artofpracticing.com/book/) (Gerald Klickstein' book is also good, i remember)

Julie Lieberman has soem good violin-specific eg [http://www.halleonard.com/product/viewproduct.do?itemid=6954...](http://www.halleonard.com/product/viewproduct.do?itemid=695420)

<comment><USER10> Galamian's book -- loved it!

> convex optimization

I assume you are minimizing -- maximizing is easily NP-hard.

So, get some supporting hyperplanes of the convex _epigraph_ and use linear programming to minimize subject to those hyperplanes as constraints. Then add a constraint and try again. There is also a cute way to do _central cutting planes_ that can be better numerically.

So, for many non-linear problems, if are maximizing, then the dual is a convex minimization, and that can help. This can be called _Lagrangian relaxation_.

I never thought that how to do violin practice was so tricky -- just work to get the fingers on the notes, in time and in tune, and then do it a lot until get a lot of facility ( _hear_ the note(s) just before playing them), sometimes return and play very slowly and deliberately, and then work on expression -- the fun part.

But then I never made much progress. The best I did was the D major section of the Bach "Chaconne", and that was fantastic fun -- I played the repeated notes that made them sound _insistent_! I regard the end of the D major section as the climax of the piece and find it fantastic.

<comment><USER9> That reminds me, somebody borrowed my Galamian and never returned it... I want to read that bit about vowels and consonants again.

I think all of this is about you need to figure out how to practice or learn CUDA or Mandarin or whatever for yourself. My dad's advice on learning math was simple "Stare at the book til something sinks in, do the problems, figure out if the answers in the answer key are right". For some students, that's all they need to be told

What this guy talks about at the bottom of page is similar, i.e. there are people who know the materials and teaching methods that have worked int he past, but the only person who can teach yourself is yourself: [http://www.allaboutjazz.com/janek-gwizdala-cooking-up-a- litt...](http://www.allaboutjazz.com/janek-gwizdala-cooking-up-a-little-bass- magic-janek-gwizdala-by-ian-patterson.php)

<comment><USER7> > "Stare at the book til something sinks in, do the problems, figure out if > the answers in the answer key are right"

In Rafe Esquith's classroom, he talks about how they take that a step further. They knock the ball out of the park on standardized testing because they specifically try to come up with _wrong_ answers for a math question. They learn the psychology of making test answers by doing - and don't fall into the traps.

[http://www.amazon.com/dp/B000OZ0NRG/](http://www.amazon.com/dp/B000OZ0NRG/)

</comment></comment></comment></comment><comment><USER11> >When she does this with students, she asks them afterwards, "Was anything fuzzy?" It's usually the part that they were messing up that's fuzzy.

This is genius. Reminds me of the rule that if something feels fuzzy _at the instrument_ it's always a conceptual error about the piece rather than poor coordination or poor finger strength.

>Why is it better to practice every day for an hour, instead of seven hours on one day of the week?

For the same reason that it's better to practice for 10 minutes several times per day instead of one hour straight.

<comment><USER12> Not the same reason, as it is better to practice one hour every day of the week instead of seven hours on one day because of the sleep you have.

<comment><USER11> Sleep is a given. The brain is learning continuously, not just during sleep. Go back to a section of a piece after 15 minutes doing something else and notice it has become easier.

<comment><USER13> Sometimes it's even easier after just 1 minute away. I code and play music, and I've made a habit of occasionally walking around the house, petting the dog, then back to where I'm working/practicing. It doesn't take long, but it works strangely well.

<comment><USER11> Yes. There's something about _walking_ isn't there? Returning attention back to the body and the senses amounts to _relaxing_ after hard concentration. Perhaps petting the dog enhances that, the body being where emotions are felt. Scientists seem to suspect that walking or just standing occasionally is important for health.

To complete the argument about learning: you play the tough section, correcting a few errors. Then you go away for a bit, and during that period the brain updates its hardware to include those corrections. The mistake is to hammer/saw away repeatedly and interfere with _what you have already achieved_ , wasting time and effort. I think the OP kinda acknowledged this by recommending switching at whim between different tasks _within_ a practice session.

</comment></comment></comment></comment></comment><comment><USER14> I found this article very interesting, but I'm not a musician. I know they mention sports briefly, but I'd love to see a more detailed account of this advice applied to other topics.

</comment><comment><USER15> I don't find this to very useful, and I think the approach of this study is probably not so great. My background on this is that I'm a professional violinist. I started playing when I was 4 and was performing professionally when I was 8. I'm 35 now, and I still play as a soloist, in chamber music, and with an orchestra. I happen to make most of my money by writing code, but the violin is still a big part of my life. I teach kids, and they win big important violin competitions.

My specific issue with this article is that it seems to think of practicing as one monolithic event--as though you are really just doing one big thing when you practice: getting better at what you are practicing. I think the concept of randomizing tasks is sort of getting at a good point, but it's really not all the way there. And also, you have to understand that you are, in fact, getting something extremely valuable out of the block practicing: the ability to focus on a singular task for a long period of time. That's a skill itself. You will never do well in a 3-5 hour rehearsal with a string quartet or an orchestra if this is not something you are accustomed to doing on a regular basis.

Here are some more details.

The big thing to do when I was a kid was to send your children to a practice camp like Meadowmount, where all the Juilliard School teachers spent their summers teaching kids like me. That was all block practice. 5-8 hours a day, and a lesson every day or so. That kind of block practicing went exactly as described in the article. An hour of scales, and hour of etudes, an hour on a piece of music, etc.

In my opinion, that's a mixed bag. You need to be able to pull a scale out and just lay it down. Any scale. Any time. It's a kind of a musical Fizz-Buzz problem. You just need to be able to do it even if it's not really interesting. Although, it's not like Fizz Buzz because scales are actually really hard. Etudes are a sort of micro-problem to solve. You isolate a particular bow or finger technique, and you spend some time every day working out that physical skill--the logistics of getting x fingers from point a to point b in the requisite amount of time.

Putting in a block of time on your musical piece is also important in itself because you have to commit that music to memory. Not just the notes and the finger motions involved, but all the little things like the bowings and phrasings and dynamics and stuff. The parts that take notes and turn them into "music."

The reason I think the study is flawed is that I think it's studying a strawman of an idea. No great players actually practice in the way that this article describes. There's this idea of a mindless slaving away that we do for hour after endless hour. That's not real. I mean, it's probably real for some people. But great players don't do that to begin with. And good teachers don't recommend this approach.

There are some popular methods of learning difficult things. There's a thing that people do to tackle a difficult bit of notes where you just practice slowly with a metronome and you gradually increase the tempo. Start with a tempo slow enough for you to get the notes out, no matter how slow. Then you just gradually make it faster and faster. This should work, right? No. It should not work. And it doesn't work.

\--a brief interlude before I explain why that doesn't work--

What you're doing when you play a musical instrument is that you're solving physical problems--usually without the necessary tools. You are working a variety of physical challnges. The logistics of moving fingers from one place to another, you're solving acoustical problems by managing the friction between your bow and your string(s). Or if you're a wind player, you're dealing with the physical properties of your breath interacting with a vibrating reed or you're vibrating your lips when you play a horn of some kind. There are two parts of every musical problem. 1. There's the conceptual problem: "How should I be trying to execute this solution?" 2\. The physical part: "How do I train my fingers, arms, hands, lips, whatever to perform this solution?"

Tempo matters when you are planning out these solutions. A slow tempo uses different physical motions, and it takes a different mindset. Slow tempos (yes, tempi, I know, but I don't feel like the word is so foreign that I need to pluralize it in Italian, so tempos) allow for larger motions and gestures. Slow tempos do not need to have efficient motions the way fast ones do.

\--back to my point--

Practicing slowly will never get you to perform things quickly. You are practicing the wrong thing. If you want to learn to play things quickly, you have to practice them quickly. But in small chunk. Take 2 or 3 or 5 notes. Play them even faster than you ever intend to perform them, and do it with a metronome, so your rhythm is good. Chain them together, and gradually slow down and enlarge your motions.

I don't know the details of this study: who were the participants, where they came from, or what their experience is. But to me, this seems like a situation where the block practicers are basically dumb practicers. Because no good musicians practice that way. I think you would have positive results doing absolutely anything to break up this kind of a thoughtless, put-in-some-time- and-hope-I-get-better kind of a routine. Random exercises are probably better than that.

I can see how these results make sense. Randomization is probably a good thing for a certain type of musician. But I would argue that it's definitely not better than a thoughtful, focused, problem-solving approach applied for long periods of time.

Anyway, just thought I would throw some things out there. Cheers.

<comment><USER16> This is a fantastic comment. I'd love to hear more about your experiences and any other insights you have about musical practice.

Since you program too, do you think there is much crossover? Some analogous form of practice that makes you a more effective coder?

<comment><USER15> Some thoughts about what crosses over:

One of the most unexpected things to me is how good I am at hitting deadlines. But it makes sense. As a musician, deadlines don't move. You're playing a concerto with an orchestra on such and such a day . . . that concert is going to happen whether you are ready or not. Your choices are to get up there and play like a badass or get up there and fail in front of a thousands of people. You learn that when you're 8, and it sticks with you. When I moved over to coding, I never thought twice about it. A deadline is a deadline. It doesn't move. That's a thing that's been consistently talked about in my career. I nail my deadlines. Not because of any magic about me as a programmer, just a mental inability to view those as flexible. I can see, however, that this would be a weakness for me if I were to get into management: I'm awfully impatient with people who don't hit deadlines.

Dealing with toxic environments. I read about brogrammers and silicon valley startups with all kinds of ego problems and sexism all the time. We all do. I've even worked for a startup or two that styled itself that way. I've never met anyone with the kind of ego that professional musicians have. Not all, mind you. It's been my experience that the best people in any field are quite kind and wonderful and humble. And the worst are the ones who are actually just mediocre. But there are tons of mediocre performers who have terrible sexist, racist, and generally toxic attitudes. I managed an orchestra while I was studying statistics after I dropped out of my music degree because it seemed like a good idea. I've never seen such a wretched hive of scum and villainy. As far as I'm concerned, even the most obnoxious of the party-boy, popped-collar, douchebag bros I've ever worked with are basically nothing compared to you run-of-the-mill regional orchestra player jerk.

Being willing to learn from anyone. There are many musicians (particularly string players) who subscribe to a certain philosophy of playing. All other methods of playing are ipso facto wrong. My best teacher is a violinist, Bruce Berg, who studied with Galamian, Gingold, and Dorothy DeLay. He did an undergrad, grad, and doctorate at the Julliard School. Where he claims to have learned very little (I doubt this is true). After finishing his doctorate, he went and studied with a Cellist--heresy!--named George Neikreug. What could a violinist with these credentials possibly have to learn from a Cellist? To this day, Bruce claims that he never actually learned to play the violin well until he studied with George. I've taken that same approach, (and I picked up a lot of George's techniques from Bruce), both in music and in technology, and I think it has served me well. Go to the dark side for advice some times. Go learn a language that you don't like. Go talk to people you don't think have anything to offer you. Go with an open mind and a warm heart. And a couple hundred bucks. Because people often charge money for their knowledge. But I've learned as much about programming from reading Marco Arment talk about how he does all his web apps in PHP because his needs are simple as I have learned from reading Eevee about how much PHP is a hammer with the claw part on both ends.

I could continue on about this just as much as the earlier topic. But I should probably shut it down. So I'll close with this:

Don't be afraid to contact big important people and ask them for help. I remember when I was just learning python, I was writing an extension for SPSS to do some statistical junk that we couldn't do with the interface as it existed. So I just went and wrote one that shouldn't have worked but did and emailed him the code and asked why it worked. I got completely schooled by a total master. It didn't really work. Slightly embarrassing, but since I didn't bring an ego to that situation, I learned a ton. John Peck is a really great guy.

</comment></comment></comment><comment><USER17> "Mental Practice" Now there lies the key to the door...</comment>


Show HN: I built a webapp to help web "designers" create logos and fonts quickly - <AUTHOR> http://use.fontorie.com <comment><USER2> Are you sure you didn't mean you created a tool to help web _developers_ create logos quickly? I'm a designer and I really can't see myself ever using this tool. I have Photoshop open at all times and know all the typefaces I have on my system, so I can bust out something incredibly simple in seconds. Plus, for logos, there's just not much benefit of using a web font since you typically won't use the same font from your logo elsewhere on the site, and the selection of good web fonts is limited and their kerning sucks and/or can't be adjusted at a precise level.

Just my 2 cents.

Edit: One more thing, designers don't create logos quickly. They're sweated over and tweaked and modified over and over until they're perfect. It's not something that we quickly generate or don't care about if there are small alignment/kerning issues. I'm sure there are some good use cases here but they don't immediately pop into my mind.

<comment><USER3> I was going to write a similar reply to flyosity. This is a tool for non- designers. Designers already have far more precise tools.

</comment></comment><comment><USER4> Very nice and useful website. Couple of points:

      * The input box for the logo name is not obvious.
      * The options on the left hand side are quite techy. I don't understand how they can help.
      * Do all the four drop down boxes on the bottom work together? Or is it one font at a time only? The default view shows four active fonts. I was thinking that there was some cool algorithm which 'mixes' fonts. Is there any?

</comment><comment><USER5> I spent a good 15 seconds trying to find the input box to type my fictional website name. You might want to make that more obvious.

<comment><USER6> Where is it? I gave up on finding it after about 30 seconds.

EDIT: Tried again, and realized the output is also the input.

</comment></comment><comment><USER7> Couple things:

* a suggestion: add a colour slider to allow changing the background colour of the page. It can be jarring whipping something up against a light blue background that's intended for a site with a different colour scheme.

* minor bug - when choosing a font from a different dropdown box (ie. picking one under "curvey", when the current selection is from "standary"), if the font is already the "chosen" one (ie. if the box was displaying "sniglet", and I click it again), the logo font doesn't refresh. It's not hard to work around (pick something else from the list, and then pick that one again), but it shouldn't be hard to fix either.

Other than that, I like it. Well done :).

</comment><comment><USER8> Great, I need something to help me design a logo easily. You could even guilt- charge me $1 once I click SAVE.

However it isn't going to work as is. You need to show all fonts immediately. As a user, and potential paid user, I do not want to click endlessly.

<comment><USER9> Agreed, being able to see multiple variations of the logo simultaneously would be nice.

I could see myself paying somewhere between $1-$5 for a zero-effort placeholder logo like this.

</comment></comment><comment><USER10> I'm seeing the logo half-cropped off the top of the screen, save to examples and the link is overlapping the Logo, and I can't find the text input anywhere on the screen to put my text into. Chrome and Firefox on Mac OS X 10.6.7.

</comment><comment><AUTHOR> The fonts are from Google Web Fonts API. Check out the examples for some awesome stuff that can be done with the site. Also add an example if you are visual talented. Finally, I'd like a lot of feedback on what needs to be done to make the webapp great.

<comment><USER11> Initially, I was under the impression that this tool generates an image. Then I realized that it uses Google's Fonts–which makes it infinitely more useful for super fast logo prototypes. I would Make this more obvious. Don't hide the final "copy-and-pastable" CSS behind a link; it's the best feature.

It could also find more varied use if you allowed the user to input more text at smaller sizes in order to generate Google Font based CSS for other elements of a layout (or to preview how a logo would look at a more realistic size).

I would suggest a slight redesign of the controls on the left so that it is clear which elements affect the font and which are purely for the shadow. Similarly, the "submission" of a shadow to the list of shadows needs a more obvious description (both verbally and visually) of how this process functions.

Overall, I think this is an excellent way of getting a (placeholder) logo up and running in 20 seconds and I'd love to see how you expand on its current state.

</comment></comment><comment><USER12> As a developer with a slight interest in design, this is great. Definitely bookmarked for future use. I can think of just one suggestion outside of what others have suggested:

I'd love it if I could get it as an image instead of CSS. Since I would only use it for a display and probably not any dynamic text, just a transparent png export of the image would be sweet. Even a non-transparent image export would be useful (and necessary with any amount of blur on the drop shadow).

On my own I can print screen and pull in the text into an image, resize it, and make it transparent but others might not have the resources or know how to do that.

</comment><comment><USER13> This is cool, i'd use it. Its quick to get an idea of what font to use. A few improvements (using chrome): 1) Didnt know I can change the text that says 'Logo.' Took me a while to click the text and change it. Indicate that somewhere. 2) It would be cool if I was able to change the background as well. 3) In the drop downs, it would be awesome to see images of sample text of that font. 4) Letter spacing would also be awesome.

That's about it for now. Keep up the good work! Looks cool!

<comment><AUTHOR> Thanks for the great feedback.

</comment></comment><comment><USER14> Would be nice if you could include some information about whether the usage of the fonts is free eg licensing/legal type information

<comment><USER15> fonts are totally free google puts them

</comment></comment><comment><USER16> I think this is very cool.

One piece of feedback from how I would use this type of tool: Once I set my logotext, which I found not to be a problem, I would love to just jam on the down arrow on my keyboard to cycle through all. It sort of worked because it would open the last select used. but then it be pretty cool for my uses at least.

</comment><comment><USER17> I've bookmarked this for myself later on- I'm a developer with no design chops whatsoever. There's a big space for apps that make it easier for developers to push out aesthetically pleasing front ends.

</comment><comment><USER18> Looks awful on this (admittedly small) screen: <http://dl.dropbox.com/u/2819425/fontorie.png>

</comment><comment><USER19> Speaking as a developer who royally sucks at design... THANK YOU. This looks like it could be a big help for me :)

<comment><USER20> SQL

</comment></comment><comment><USER21> Hey, I'm getting an error:

        http://use.fontorie.com/5d90df57ffffffkkf000efn

<comment><AUTHOR> It's not prefect yet.

</comment></comment><comment><USER22> _Show HN: I built a webapp to help web "designers" create logos and fonts quickly_

FTFY

</comment><comment><USER23> I didn't see a font size option? Did I miss it?

</comment><comment><USER24> The layout looks broken on an iPad2.

</comment><comment><USER25> This is awesome. Thank you.

</comment><comment><USER26> I can't ever see a real designer using this.

A developer with no artistic skills? Perhaps.

</comment><comment><USER27> Nice until Google cuts it off and starts charging for it -- like they did with translation =\</comment>


Will your new MacBook crash to the ground without MagSafe? (Yes.) - <AUTHOR> http://www.macworld.com/article/2895694/will-your-new-macbook-crash-to-the-ground-without-magsafe-yes.html

<comment><USER2> I hope Apple develops a USB-C cable with a MagSafe decoupling point.</comment>


Circuit Breakers in Elixir - <AUTHOR> http://blog.rokkincat.com/circuit-breakers-in-elixir/ <comment><USER2> So what is the benefit of the iTunes fuse example over a very simple

Try iTunes

Catch exception

Try local search

I guess in a bad state it makes your results faster. How often does that need to happen to justify the complexity?

<comment><USER3> My thought was the complexity would be worth it if you have a non-trivial number of users.

Additionally some third party API's have rules where they expect you to back off exponentially if there are errors. You'll get your account suspended if you don't play by the rules.

<comment><USER2> I think the key is you need a backup for the system you are gating off. If your db breaks you blow a fuse and show errors... Not sure you gained much.

<comment><USER4> There is a subtle but important difference:

In a timeout situation, you know relatively little.

If the circuit tells you it is broken, you know that the backend system is deemed down and trying again right now is probably futile.

Each client for a server (on which the fuse runs) will be rather blind to the experiences of _other_ clients. But the server, and thus the fuse, will know what happens to all the clients. This allows you to act globally on the server side. But this in turn empowers each client locally since it now also has global information.

<comment><USER2> So making a fuse global, say throwing it in your db, now means you can't have a fuse about the DB. Which may be okay, but seems in some ways less good than a local fuse.</comment></comment></comment></comment></comment>


Solving Rings in Agda - <AUTHOR> https://oisdk.github.io/agda-ring-solver/README.html <comment><USER2> Whenever I read a technical overview like this, I like to see references to existing literature. This report doesn't do this, see for example [https://link.springer.com/chapter/10.1007/11541868_7](https://link.springer.com/chapter/10.1007/11541868_7) which is basically the same thing but in Coq.

<comment><USER3> Author here! It's my fault the code and report are a little disorganized, but there is a related work section in the full report ([https://github.com/oisdk/agda-ring-solver- report/blob/master...](https://github.com/oisdk/agda-ring-solver- report/blob/master/report.pdf)) which mentions that paper.

Also, if you click through to the section on the algorithm itself ([https://oisdk.github.io/agda-ring- solver/Polynomial.NormalFo...](https://oisdk.github.io/agda-ring- solver/Polynomial.NormalForm.html)) you'll see a link to the same paper.

</comment></comment><comment><USER4> I'm happy this exists, and that Agda has the facilities to make it feasible, but sad that automated formal systems are so primitive compared to software development tools. Is this gap just lack of motivation for more people to contribute, or the culture of formal systems, or some source of technical barriers, or ??

</comment><comment><USER5> How one can type that ℕ and ℤ on the regular keyboard?..

<comment><USER6> in atom plugin for agda you just typing \bn or \bz , full reference here: [https://people.inf.elte.hu/divip/AgdaTutorial/Symbols.html#b...](https://people.inf.elte.hu/divip/AgdaTutorial/Symbols.html#blackboard- bold-letters)</comment></comment>


Debugging the Linux kernel with JTAG - <AUTHOR> http://www.eetimes.com/design/embedded/4207333/Debugging-the-Linux-kernel-with-JTAG <comment><USER2> What are good debugging and development tools for a novice to use to get experience with Linux on a device with JTAG, such as one of those popular old Linksys routers? (perhaps using openwrt source as a starting point)

I'd like to use one that has hidden USB ports (like the T-mobile version of the wrt-54g) to make the printer logging output of another device accessible through the router web server cheaply and with minimal space/power requirements. The printer is fed serial data, as found with gas pump printers.</comment>


Lua in Lightroom - <AUTHOR> http://www.sauria.com/blog/2008/10/09/lua-in-lightroom/

<comment><USER2> By itself Lua is very fast and lightweight. I simple love how you can use it's tables or hash-tables if you will to construct complex data structures. I loved it even to a point of trying to use it to write a simple web application.

Unfortunately Kepler(a Lua webframework) is not only hard to get running but it's docs need a lot of work.

</comment><comment><USER3> Many games use lua to handle its logic too. I think I may spend some time to learn lua. However, do you guys think that lua will be an universal scripting language like python or not?

<comment><AUTHOR> It works fine for scripting general things, it's just meant to easily interface with languages like C.</comment></comment>


The importance of decentralisation - <AUTHOR> https://ungleich.ch/u/blog/the-importance-of-decentralisation/ <comment><USER4> It's sort of an ongoing hobby of mine to study the forces of decentralization vs centralization.

This article about decentralization does what many other evangelism articles do: _talk about the ideals and benefits_.

However, I believed what's rarely discussed but more important is the _economic forces_ that prevent decentralization from fulfilling the idealists' vision. Yes, decentralization will be in effect for niche groups but I don't see it becoming mainstream.

To purposely be provocative to spur discussion, I will make a bold claim: _Decentralization is an unstable equilibrium. It 's the centralization that becomes the stable status quo._

If tpcip protocol and http protocol are _already decentralized_ , why do we have centralized services that have "too much power" such as Facebook/Google/Youtube? It's because different actors can _spend more money_ on their particular http node than other http nodes. Those _unequal economic forces_ is what makes decentralization tend towards centralization. There is no technical protocol specification that can prevent that.

E.g. if Git protocol is decentralized, why is there so much concentration on Github? It's because John doesn't want to install a git server on his laptop and punch a DMZ hole through his home router and leave his laptop up & running 24 hours a day to serve up his git repo. He'd rather spend the weekend playing with his children. And Jane doesn't want to spend $30 on a Raspberry Pi and install Gitlab on it to serve up her git repo. Multiply John and Jane's by a million other devs with their own various reasons for not serving up their git repos in a decentralized manner and the emergent phenomenon you get is something like Github.

See the trend? Centralization is a _natural outcome_ of millions of people not wanting to (1) spend money and (2) not wanting to spend extra time -- to fulfill ideals of decentralization.

I wish we would discuss the above factors much more often and there were more articles about it.

<comment><USER5> Nature and Biology gives us a very strong counterexample to this. If centralization was the stable form you would expect to see an overwhelming preponderance of it. Instead you will almost always see a blend of centralized and decentralized forms, with a bias towards decentralization wherever it's a good fit. The vascular system is centralized; the nervous system is mostly federated; most of the rest is highly decentralized. A counter argument is that intelligent design changes the relative value of these different forms but I'm not sure whether there's enough evidence to support this.

My theory on the matter is that the trend towards centralized forms in our current society has a lot to do with Conway's Law (organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations.)

We live in a society that currently has a very hierarchal, centralized and structured systems of power. This means that the subsystems (eg Corporations) and the artifacts of those systems are subtly pushed towards a similar form.

<comment><USER6> I would argue that there's an incredibly powerful force that drives the consolidation and centralization of systems in modern society: the wholly artificial legal concept of property. We are given a legal right to accumulate control over as large a swathe of material goods and organizational structures as we can within the confines of the law, and this ownership is backed by the promise of state violence should it be violated.

The concept of property, and the fact that our laws and government are built around protecting it, is the sole mechanism through which massive disparities in power are achieved and maintained in society. It centralizes wealth through avarice, making our stomachs unlimited in size. It also creates the dilemma of externalities, as we differentiate the value to us of the things we "own" versus the things we hold in common with others or that are "owned" by the state.

Property is also a entirely artificial concept. It takes the practical concept of use and possession and extends upon it to completely absurd extremes. It ties our society up in knots wasting a massive amount of human energy in the activity of determining and enforcing property ownership, and leads to incredibly convoluted structures in laws and markets, often leading to huge structural failures.

For instance, intellectual property law is completely absurd, on every level. It is an effort to force something that is fundamentally free and infinitely available to everyone to become scarce exclusively so that we can properly ensure it fits into a property-based system. The amount we lose as a society from the fact that all software is not simply open source is incalculable.

<comment><USER7> _> wholly artificial legal concept of property_

Property is an informational concept more than a legal one. The concept of "mine" is present at a very young age, and clearly present in various animal groups. The data-structure is a simple association of objects with an owner, and for the substrate animals use only their brains, but humans use paper and, increasingly, bits. Banks and local government agencies keep the most official of these records, but informal records count too. (My son may think my daughter's blanket is his, but she will protest loudly and vehemently, and there is not really a formal reason for her claim).

 _concept of property...is the sole mechanism through which massive disparities in power are achieved and maintained_

Property is a necessary, but insufficient condition for inequality. Property is how you keep score; the reason some players win far more than others is not entirely known, but I'm pretty sure it has more to do with personality, talent, and luck than the jurisprudence of property rights.

It's not clear what a society without property would be like, BTW.

<comment><USER8> The "natural" concept of property is much more narrow than our abstract notions. Animals, and human societies before they develop the abstract property theory, generally define property through possession or use - <USER2> house is yours because you live in it, that sort of thing. You can't establish an association simply by saying, "this is mine". This puts a natural limit on how much property can be accumulated.

Once you have abstract property as a concept, it's all about tracking the association in and of itself, separated from any physical manifestations of it (possession and use). Your house is yours not because you live in it, but because there's a title on it in your name, and a record in the registry. This makes it possible for one person to have ten houses, live in one, and rent the rest out to those who don't have any. If they accumulate the rent, they can eventually buy more houses, and it becomes a positive feedback loop.

But there's no particular reason why any given society has to adopt such a system of property, and not all societies did or do today - especially with respect to land and other immovables. This can often be seen in the colonial history, where the colonizers expected to be able to buy and sell things that the indigenous population didn't even consider property, or at least the kind of property that could be bought and sold. More often than not, the colonizing power would then declare that in the absence of a "proper" abstract property claim on e.g. the land, it's there for the taking of anybody who is willing to make one. And that is how we ended up in the present situation, where that system of abstract property rights is mostly enforced throughout the globe - but I don't see how it follows that it's a system that is inevitable for any human society.

<comment><USER9> "But do you have a flag?"

</comment></comment></comment></comment></comment></comment><comment><USER10> The article calls ipv6 an "easy way" to decentralize but I don't see it.

If you click the link on how to get your own ipv6 space you get two options: Ask your ISP for an address or set up a tunnel to someone else who will give you one.

These are the same options we have today for ipv4. I went on my crappy large ISP's website. They are not handing out static ipv6.

Why would they? The problem has always been more about corporate power than tech.

<comment><USER11> That'll change with time. I'm behind ISP NAT for IPv4 but have static IPv6 direct to all my devices in the home. My ISP ran out of IPv4 relatively early so had to go this way.

On the other hand, this won't really change much for the firewall issues named above. Computers are too insecure to expose them all to the internet, all the time. We'd need pretty radical changes in how software is written to create a truly flat worldwide network (which has never existed before).

</comment></comment><comment><USER12> I may be somewhat confused, but how would a different IP address/system prevent centralized services? It seems to me that the same market problems, the strong vendor lock-ins explained in the post, will still persist in IPv6. There's no mention of how/why IPv6 solves those problems, just that they do...

<comment><AUTHOR> It doesn't prevent them, but with IPv6 everyone can take back control. With IPv4, you are unable to get a decent amount of public addresses to run your service (if you are not having tons of money already).

<comment><USER13> You are talking about lower layer protocols here that don't matter for decentralization. In fact, a decentralized transport protocol can be made that doesn't require your ISP to give you an internet routable IP address nor put you behind NAT.

Think e-mail, for example. It never required you to have an internet routable IP addresses to communicate with anyone. There were and are plenty of local networks where people run local SMTP servers that communicate with upstream SMTP servers over local network and only those have public IP addresses to communicate with SMTP servers over public internet.

</comment></comment></comment><comment><USER14> [https://github.com/awesome-selfhosted/awesome- selfhosted/](https://github.com/awesome-selfhosted/awesome-selfhosted/)

<comment><USER15> If you want to selfhost many of those apps, we have built a tool (Cloudron) to take away most of the deployment hassle for many of those apps mentioned in the link. Also to work around the provider lock-in with ipv4 addresses while hosting from your home or on-premise we have a built-in dynamic DNS feature [https://cloudron.io/documentation/networking/#dynamic- dns](https://cloudron.io/documentation/networking/#dynamic-dns) So in the end it doesn't matter if you change ISP or if you don't even have a static ipv4 as such.

<comment><USER16> Cloudron is missing apps for most use cases I actually care about.

Can you add a google photos type app? Maybe Piwigo (it has a good mobile app) or OwnPhotos[1] (it has face/object recognition). Lychee doesn't have a mobile app or face/object recognition.

Can you add some communication apps like Matrix Synapse or Ejabberd? Mattermost is walled off from the outside world.

Can you add some social apps like Pleroma or Mastodon?

[1]: [https://github.com/hooram/ownphotos](https://github.com/hooram/ownphotos)

<comment><USER17> Well, Cloudron provides apps for most use cases _most_ people need. So the fact that you need some obscure apps doesn't invalidate Cloudron as a great solution. I've been using it for a long time and love what it has to offer. Plus the devs are great and keep working on making it better.

</comment></comment></comment></comment><comment><USER18> Local-first software development[1][2] could also be a force for decentralization, since you don't need a coordination server to hold the authoritative version of a document. The relevant protocol for discussion is the CRDT (conflict-free, replicated data type).

[1] [https://www.inkandswitch.com/local- first.html](https://www.inkandswitch.com/local-first.html)

[2] [https://news.ycombinator.com/item?id=19804478](https://news.ycombinator.com/item?id=19804478) (6 months ago)

</comment><comment><USER19> Some random thoughts on centralization vs decentralization. In theory, IPv6 allows every human, pet, alien and robot to have an address. Each address can host let's say the following services

1\. info - <USER3>

2\. data transfer - ftp/http/p2p/...

3\. communication - email/voip/...

4\. entertainment - game server/media server/...

Right now the biggest problems with decentralisation are

1\. ownership of hardware

2\. assigning of address and interfacing with the network.

3\. configuration and setup of services

4\. scalability of the service

5\. using decentralized stuff for illegal activities

I think 2 and 3 is the biggest win improving decentralized services and getting rid of facebook and the like. Why are two and 3 still hard? This will also make decentralized services appealing to normal folks. The biggest challenge to making decentralized services mainstream is 5.

Hardware is manufactured by a few monopolies and they impose some restrictions and tracking abilities. The assigning of an address is done by the telecom and they have some rules, regulations and tracking abilities. Each country right now is setting up new rules and regulations for the data that comes into it. As configuring and the setting up of stuff is hard we once again have a few major services. The illegal activity makes decentralized services seem like the wild west and centralized services seem like stable societies. The spirit of legality is to ensure fairness but when that doesn't happen people turn to decentralized mediums to express themselves.

</comment><comment><USER20> I wrote an article about just the e-commerce aspect of this a few weeks back:

[https://battlepenguin.com/tech/the-death-of-the-mom-and- pop-...](https://battlepenguin.com/tech/the-death-of-the-mom-and-pop-e-store/)

There are fewer and fewer websites with things for sale. Everyone puts their stuff on an Amazon, eBay, Newegg, Etsy or Reverb store. The big carriers are the store-of-stores.

Some people setup a Shopify or other site as well, but it's usually secondary, or it's to sell merch for another thing like webcomic or blog.

</comment><comment><USER21> As someone who works in the domain name industry, I'd like to note that there is ever increasing pressure to police domain name use. When you buy and use a generic TLD such as "com" or "ninja", you are buying a US product and it is subject to sanction laws and other things you may not realize. CC TLDs (two characters long like "ch") are products of that country and have their own laws UNLESS the registry company is located in the US - in which case it is also subject to sanctions.

I point this out because unless you get everyone using your IPv6 address directly, your name is certainly NOT decentralized.

<comment><USER22> Any thoughts on which TLDs are least susceptible to shenanigans?

<comment><USER23> Decentralized ones like provided by OpenNic

[https://en.wikipedia.org/wiki/OpenNIC](https://en.wikipedia.org/wiki/OpenNIC)

</comment></comment></comment><comment><USER24> Is there an effective way to suppress denial of service attacks in a decentralized network?

Given that there are DNS root servers, isn't the internet actually centralized?

<comment><USER25> You have to centralize somewhere—IP address assignments are also centralized.

<comment><USER26> Not really -- look at CJDNS, Tor, or IPFS for examples of totally decentralised addressing schemes. These can be mapped to human readable addresses through Namecoin or Ethereum Name Service. We do centralize somewhere, but we don't _have_ to.

</comment></comment></comment><comment><USER27> So long as ISPs are centralized, it doesn't matter if everyone has their own IPv6 address because ISPs are gatekeepers that can censor requests.

</comment><comment><USER28> The Internet Archive (parent of the Wayback Machine, Top 300 website in the world) is doing a lot to help re-decentralize the web.

Check out this post for more info:

[https://news.ycombinator.com/item?id=17685682](https://news.ycombinator.com/item?id=17685682)

</comment><comment><USER29> This underlines the positive side of Tall Poppy Syndrome. Envy has a number of serious down sides, including making the sufferer miserable for, usually, very little return. But when people stop using an Amazon, Google, Microsoft, etc., _because_ they are successful, it creates a counter-force to centralization. Like hate, when well directed, envy can be good. Like hate, it can easily get out of control and cause great damage, up to and including genocides.

<comment><USER30> Envy might be a factor but I feel like wanting freedom and mistrusting authorities/corporations are stronger here.

</comment></comment><comment><USER31> Sorry for the naive question but can anyone explain why hosting an IPv6-only server can help with decentralization? As a provider of some service, say you have a website where you publish some work, is there any advantage of setting it up as an IPv6 server?

<comment><USER32> Yes and no.

You can build centralization on every layer of the network stack. Facebook is centralization on the application layer. The problem is that with IP addresses being as rare as they are in the ipv4 space you have a lot of tricks that you need to use and ultimately you go through someone using a real ipv4 address.

Ipv6 doesn't have that, so it lets you decentralize further down the stack. While this is good in its own right, currently we don't have issues with ip address monopolies, unlike application layer monopolies.

And that's the problem we have. Doesn't matter how many open layers you have, someone can always build a closed system on top of them.

</comment></comment><comment><USER33> So what's the latest deal with IPv6?

IPv4 was supposed to run out and the Internet come crashing down 5 years ago, yet here we are, and everything seems just fine, while a billion smartphones were added.

<comment><USER34> > while a billion smartphones were added

A huge number of them being on IPv6-only mobile networks using NAT64 to access the old Internet.

</comment></comment><comment><USER35> Missed a really cool vanity url opportunity :)

</comment><comment><USER36> The internet will not stop being decentralized just because there are some consumer product monopolies.

When Amazon starts running its own dark fiber, and you can only use that fiber to buy shoes that are only sold on Amazon, then _part_ of the internet will be close to being centralized. But that would still just be a small part of it, and it still won't happen entirely.

There's a very long tail between Amazon and the user. How does Amazon connect to the user? Sure, it starts in their datacenter. But then immediately they need to connect to multiple points of presence, which means multiple bundles of fiber going in different directions. And so you'd say, sure, Amazon has lots of DCs, so they could just run dark fiber between all of them. But they're not _everywhere_, so they need to eventually peer to a more global network.

Eventually you get to the ISP. There's two kinds of ISPs: wired and wireless. While they're increasingly the same company, there is a wealth of technology, expense, competition, and physical infrastructure wrapped up in each. Copper and fiber runs to every home, customer support, billing, management, contracts with public and private entities. There's multiple companies in these industries that are bigger than Amazon.

Say Amazon becomes its own ISP. They can either run fiber to every home (lol) or become their own nationwide wireless ILEC (lol) or they can become an MVNO and rent access to an ISP's gear (possible) or they can just pay internet backbones to peer with them and get access to all ISPs' customers. The first two would basically be like making a brand new Comcast; uh, good luck with that. The third is what Comcast already does: they rent ILEC's networks to provide their own mobile service, capturing more customers. The last is how the internet operates today: the ISP deals with the complexity of getting everyone in the country online, and Amazon just pays to connect to POPs.

In a non-net-neutrality world, any ISP can add a line-item to your bill for you to get access to Amazon. In that case, Amazon becoming an ISP avoids that, capturing more profit in the process. But why in "Bob"'s name go through all that work, when you can just charge people individually to access your website? Aka, Amazon Prime. So in order to completely control your access to shoes, they can either build an ISP, or just..... use existing ISPs. Currently, most ISPs aren't adding line-items to access Amazon, so the latter works fine.

In another bizzaro possibility, Amazon could merge with every ISP in America, creating a hyperconglomerate, so only Americans could access Amazon, and every ISP bill is charged for Prime, and every non-American ISP has to pay to route traffic to Amazon. I think that would just crush Amazon's sales, but it's possible. But _still_ the internet would be decentralized, at least globally.

And one final option that actually already exists in developing nations: Amazon and a handful of other monopolies subsidize ISPs to create a "bare- bones" internet plan, where you literally can only surf to Facebook, Amazon, and Google, but you only pay $10 a month. _This_ would be a consumer-only internet that is totally centralized and monopolized - but it's still not the whole internet.

Then of course there's _every other business in the world_ that is not consumer-oriented, all of whom depend on the internet for their business. They also have an interest in a decentralized internet, because it helps them compete with each other, too. They'd be happy to fund a decentralized internet, if only for themselves.

This is all besides the fact that decentralization is actually an architectural decision _made by a central organization - the DoD_. They made it decentralized because it just works better, not because they wanted the whole world to hold hands and sing kumbaya. Even in this fantasy world of a centralized internet, with one company managing all the consumer services, b2b services, and internet connections, they'd _still_ keep a decentralized architecture, because they know it's really friggin' robust. The network architecture has nothing to do with who has control.

If your concern is monopolies, an internet protocol does not change this at all. If your concern is being able to host your own services, that's still at the discretion of your ISP.

</comment><comment><USER37> this article doesn't deal at all with DNS so kinda misses like the biggest centralization flaw on the internet but whatever</comment>


Dear Email Industry, We've Got a GDPR Problem - <AUTHOR> https://www.jacquescorbytuech.com/writing/gdpr-email-tracking.html <comment><USER2> "This stressed out a lot of email marketers, who quite rightly realised that the new regulations would have a significant effect on their ability to acquire and market to customers via their email address"

"The overwhelming majority of commercial email sent today contains tracking pixels and tracking links, these are used to uniquely identify individuals so that opens and clicks can be correctly attributed to them"

Good.

While spammers may have a problem, people don't.

If I want your tracking pixels and emails then I'll opt in.

<comment><USER3> > While spammers may have a problem, people don't.

Marketing spammers maybe, but now scammers and malware spammers have the floor instead. Laws only stop the law abiding citizens from doing their thing, it sure doesn't stop the criminals from.... being criminals.

<comment><USER4> Not having marketing spammers is still better than having spammers and scammers

<comment><USER3> There's usually an unsubscribe button for marketing spam, and if there isn't I usually block their email. You can't do that with scammers / illegal spam.

<comment><USER5> Have you ever clicked one of those? I've never been sure if that wouldn't have worsened the situation by giving feedback that this is an active mail address managed by somebody.

<comment><USER6> If it's from a legitimate company in American jurisdiction they are legally obligated to stop sending emails if you click unsubscribe. I suppose that piece of information has some nonzero value that you are giving up in exchange to not be contacted by that company.

If you filter just a single address that address can change. If you filter their domain you might lose legitimate correspondence.

<comment><USER7> I've had several groupings of unsolicited marketing emails over the years where I've clicked Unsubscribe and ended up on what's very clearly a Totally Not That Email List, Honest...but it's advertising the same things, in the same way, just from a slightly different email and possibly different company name. They have all been American in origin.

<comment><USER6> You can complain under the CAN-SPAM Act.

<comment><USER8> It's so asymmetrical though. The amount of effort it takes to spam someone is vastly lower than a complaint.

What would be great is a third-party site where you can somehow document/log unsubscribe requests. Then, if the company still spams you, document that. A few hundred users is pretty good proof, and the company can't just argue a glitch. It'd pay for that.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> Since the headline misled me, this is about the email marketing industry and their use of tracking pixels, as opposed to issues with generic emails.

<comment><USER10> This isn't bound to email marketing, unless you put all commercial communication dealt via email as email marketing.

<comment><USER11> I think any email that uses tracking pixels could be classified as either spam or marketing, or possibly both.

If you use tracking pixels in your receipts, I think you're doing it wrong.

<comment><USER12> What about mandatory service announcements where you want to know if you reached clients before breaking them?

<comment><USER11> Have them click a link to acknowledge receipt.

A tracking pixel hit only means that that the email was received and loaded in some email client, not that it was read (in detail, or at all), understood, or acted on.

<comment><USER13> Moreover, a lack of tracking pixel hit doesn't mean the e-mail wasn't loaded, much less that it wasn't read, understood or acted on.

</comment></comment></comment></comment></comment></comment><comment><USER14> Browsing the web in Europe is like experiencing the rebirth of the pop-up ads era. It has lead to compulsory acceptance. This too shall pass.

There is a reasonable expectation that when you submit your email to a company in exchange for their service, they will email you communications relating to their products and services.

<comment><USER15> > _There is a reasonable expectation that when you submit your email to a > company in exchange for their service, they will email you communications > relating to their products and services_

Certainly not. If I didn't check a box saying "I want to receive commercial emails related to your products and services" I expect not to receive those. I might unsubscribe from the whole thing if I don't have any other means of avoiding those useless commercial emails.

<comment><USER16> I generally report such "you bought something and so we signed you up to the mailing list" activity as spam in Gmail, and if the unsubscribe button links me to a third party vendor, list "I never signed up for this list" as the reason.

</comment></comment></comment><comment><USER17> There is the Email Industry - that is the problem.

<comment><USER18> Just because you associate emails with spam doesn't solve the problem of every charity, business, church, school and group needing to communicate with large numbers of email subscribers. Like anything, bad actors make it worse.

<comment><USER19> Somehow all those groups managed to get by without email in the past. I'm sure they would be fine today as well.

<comment><USER18> Just because you don't want to hear from anyone via email doesn't mean _I_ don't.

<comment><USER19> That's fair. I'm guessing those groups can all manage to communicate with you without spying on you.

</comment></comment></comment></comment></comment><comment><USER20> Off-topic: I absolutely love the design choices made by the author of this site. It's the basic browser stylesheet with some nice refinements. A real triumph of minimalism and incredibly readable.

<comment><USER21> Font is too big.

<comment><USER22> It's very pleasant on QHD. I agree that they probably should have a different sheet for lower resolutions.

<comment><USER23> In principle, using resolution-independent units like points should be fine; it's using px instead of pt that leads to problems.

(And the site is using px; it shouldn't.)

<comment><USER24> You are incorrect. All CSS units are defined in a resolution-dependent way. (I think there has only every been one exception to this, an experimental unit `mozmm`, now discontinued, that attempted to be resolution-independent, representing one physical millimetre.) On screen, the px unit is king, being defined however the device chooses to define it—most commonly one or two device pixels. All other units are defined in terms of it: 1in = 96px = 72pt, _& c._ On print, the ratios are the same, but physical length units actually have _meaning_ now, corresponding to physical measurements—well, _maybe_ they do; in practice browsers play fast and loose with it all, second-guessing the website's stylesheets all over the place, which is normally a good thing for users because few websites take care for print stylesheets, but is utterly debilitating if you actually care and want precision.

Now the question of what the root font-size is (a unit I like to call "browser em" or "bem"—I've never heard anyone else give it a proper name)— _that's_ a much more interesting question. It's almost always 16px (I have no stats ready to hand, but I'd suggest >99% of page views), but there are devices out there that have other values, mostly between 13px and 19px, and you can change the value in some browsers also. However, website layouts commonly break if the value is not 16px, if the font sizes are based in bems and media queries in px, or font sizes in px and media queries in bems, and the developers have assumed 16px (which is completely normal). The ideal situation is to use either px everywhere or bem everywhere.

In theory, using relative units everywhere is potentially nicer. In practice, you're fine using pixel units everywhere.

But 24px is still way too big.

<comment><USER23> I'm sure you're right - I am only ever a front end CSS dev in an emergency, and then only for desktop. That said, when I see a relative discrepancy between sizes on different media, px vs pt is where I'd look. That is, two things that might e.g. look the same size on a desktop browser but look different sizes on mobile.

</comment></comment></comment></comment></comment></comment><comment><USER25> It's been over a year now and so many sites are not in compliance. I'm surprised the EU doesn't start collecting fines from companies like Yahoo and TechCrunch (and all oath sites). Just two that come to mind that are blatantly violating the gdpr with absolutely no way to not consent to their tracking. Mass email spammers are another issue. Why isn't the EU collecting this money from these large orgs that are clearly in violation? It could do a lot to help the people here.

</comment><comment><USER26> I think the GDPR is a good opportunity to reflect how much tracking we really need.

Tracking has become so ubiquitous, it's become the default to put Google Analytics on a site, to put a tracking pixel into every email, to personalize every link we send out...

But so much of that tracking isn't really necessary. I've stopped tracking website visitors and stopped including tracking pixels in emails a few years ago, and nothing has really changed.

So, I guess I won't know if 10% open my marketing emails or 50%. But who cares? I wouldn't even know what to do with the information anyway. I'd rather focus on making my product better.

<comment><USER27> GDPR wants 0 tracking. That's wrong too, the internet can't work that way, even governments can't work that way. EU wants advertising to go back to the popup / animated gifs & flash / interstitial era to maximize clickthroughs in the off-chance one of them is actually interested in your ads. That's regression

<comment><USER13> GDPR wants 0 tracking _without explicit, informed consent_. That's the key thing in this regulation: informed consent. Dealing with people fairly.

> _EU wants advertising to go back to the popup / animated gifs & flash / > interstitial era to maximize clickthroughs in the off-chance one of them is > actually interested in your ads._

Not true, unfortunately. EU wants the ads to not track people without their explicit, informed consent. GDPR isn't an anti-advertising law, it's a data protection law (says so literally in the name).

> _That 's regression_

No. That's _remission_.

<comment><USER27> \- users could always install an adblocker if they dont consent.

\- users could consent once for each tracker if thats what the law cared for. Consenting for each tracker x for each website is purposeful obstruction in order to make advertising optional

<comment><USER28> > \- users could always install an adblocker if they dont consent.

\- To consent, one must be informed, so the sites would have to advertise adblockers, why they exist and how can they be used.

\- Current adblockers rely on volunteers compiling lists of ads, and sites trying to evade those lists. That's not a reasonable way to ensure a legal right, so sites / networks would have to publish those lists themselves.

\- The GDPR is about way more than website access tracking, so you'd still need all the same rules about the rest of the use of personal information. Seems like a duplication of effort and complexity.

> \- users could consent once for each tracker if thats what the law cared > for.

Just because I'm OK with a network knowing I visit nytimes.com doesn't mean I'm OK with them knowing (and using the information) that I visit pornhub.com. Consent per site is crucial.

</comment></comment></comment></comment></comment><comment><USER29> I thought GDPR required tracking to be opt in? I can't see how tracking pixels on emails comply at all.

</comment><comment><USER30> While conducting a GDPR review I discovered that our email service provider (Campaign Monitor) was logging IP addresses of our list members associated with each email open. My jaw dropped when I noticed that they were doing geo- ip enrichment, so that I could drill into any subscriber, see a history of their opening of our newsletters, and a map of their approximate location. I could see if "Bruce" was in Melborne or Petaluma on April 23rd. That kind of data is straight up dangerous and would be very hard to justify on a Legitimate Interest Assessment. That said, I haven't found a way to disable or purge that data thus far, and have been having a hard time finding an ESP that doesn't log IPs for its open tracking. We legitimately need open tracking, but certainly not with non-hashed IPs exposed. Realistically, just overall open rate reporting would suffice for our use case, not tracking of individual list member's activity.

</comment><comment><USER31> Personal opinion: Pictures and HTML have no place in email. Full stop.

</comment><comment><USER32> True that the European user should be able to opt-out just from tracking. Our platform MailUp (ESP) is handling this in the preference center (that can be customized). Here is a sample: [https://updates.mailup.com/frontend/preferencecenter/363734/...](https://updates.mailup.com/frontend/preferencecenter/363734/5533444f-58b9-46f6-9913-9097dbd8b62c/98/13414/)

<comment><USER33> By default the European user should be opted _out_ of tracking

<comment><USER5> This and only this is a valid, legal solution according to the GDPR.

Especially all the "by continuing to use our site, you'll agree to getting the shit tracked out of you"-messages are highly illegal, because the GDPR requires explicit consent.

Sadly there have been no big legal cases up till now. But the time will come.

</comment></comment></comment><comment><USER24> > This might be one of the very few instances in which I'd recommend SFMC

What is SFMC?

</comment><comment><USER34> GPDR missed a massive opportunity to standardize encrypted email. Instead we're now stuck with crappy 3rd party "secure mail" systems.

I have a startup in Denmark, and the incubator we're part of applied for an EU funding scheme. The bureaucracy for these programs is out of control, and there are claims out there that 90% of state innovation funding is blown on administration.

Long story short, I had to fill out some timesheets, and because of GDPR print out the sheets filling in everything except the personally identifying information, and then fill the rest of them out with a PEN.

</comment><comment><USER35> My friends in IM told me I was stupid for not "building an email list" for my high traffic website.

Still see no good reason to do it.

</comment><comment><USER27> GDPR is europe's problem, not of the entire "email industry"

<comment><USER36> It is a problem when you are targeting users from Europe, which you are doing almost certainly.

<comment><USER27> We already have more than enough cookie popups and "heads up" emails whenever a company changes a comma in their ToS. GDPR is a bureaucratic madness and not something to be imitated.

Want to educate users about privacy? do it with extensive educational campaigns, not by ruining everyone's experience on the web

<comment><USER37> It's not GDPR's fault that websites have awful user experience. If they really cared about privacy then they wouldn't use popups that required multiple clicks to remove tracking cookies.

<comment><USER38> If you have to untick boxes they're not in compliance with GDPR. If the button to not have tracking is gray and "accept all" is green, you're not in compliance. Many websites deliberately try to make it harder to opt-out, which is directly against the GDPR.</comment></comment></comment></comment></comment>


How to make MongoDB not suck for analytics - <AUTHOR> https://www.scaleapi.com/blog/athena#asdf <comment><USER2> I tried using MongoDB for the customer-facing analytics of a large e-commerce marketplace. It didn't work very well. The problem is that at some point you end up wanting joins.

MongoDB was actually the third try. My first two attempts were BigQuery and Keen, neither of which worked out because they support only one index - time. Users want to slice and dice by various axes! And there's an obvious additional index you need - "merchant" \- which column stores usually say propose setting up isolated partitions for. If you do that, you can't ask questions across the whole system!

We ended up with Postgres. It was actually faster than MongoDB for simple aggregations, and joins made it much better/faster for complicated queries. Of course it only works quickly if your dataset fits in RAM, but terabyte-size instances are pretty affordable and give you a lot of headroom.

That was a couple years ago. I don't know what they're using now, probably the same. It was a frantic few weeks figuring out what was going to work - each of those systems made it to production and quickly discovered to be inadequate in vivo. If you're in a startup, even if you're using exotic NoSQL systems like Google Cloud Datastore or DynamoDB - just use Postgres or MySQL for analytics. It will work long enough for you to figure out something else when you need it.

<comment><USER3> You are completely contradicting yourself.

On one hand you complain about using technologies before you have done a prototype and evaluated the product. Then you blindly tell startups to just use MySQL/PostgreSQL without having any idea of their use case or whether it matches their query patterns.

If you are a startup the right way to go is to document your use case, understand what queries those use cases demand and then find the right database that satisfies it e.g. don't pick MongoDB if you are doing lots of joins and don't pick PostgreSQL if you are doing wide-table feature engineering type analytics.

Right tool for the right job.

<comment><USER4> Without wishing to put words in their mouth, I think parent poster's point might be that PostgreSQL will do at least a _decent_ job at most things you'll want to throw at it.

This is not the case for most of the NoSQL databases where you'll pay for lack of certain features either by a) having to write a lot of code, or b) bad-to- crippling performance for use cases it wasn't meant to solve.

So, unless you're already _very_ clear on what your exact use case is going why the spend time analysing before even getting your project off the ground?

<comment><USER5> >This is not the case for most of the NoSQL databases where you'll pay for lack of certain features either by a) having to write a lot of code, or b) bad-to-crippling performance for use cases it wasn't meant to solve.

Can you give a common example of these? This article is referring to issues related to row vs column data stores, not sql vs nosql.

<comment><USER2> Having implemented effectively the same customer-facing analytics problem in BQ, Keen, Mongo, and Postgres, I'll tell you specifically:

* Column stores like BQ and Keen don't let you efficiently slice and dice data by factors other than time. If you're slicing by customer or product, your queries become incredibly slow and expensive. You start writing hacky shit like figuring out when your customer's first sale was so you can narrow the time slightly, but that barely helps.

* MongoDB doesn't do joins. So you denormalize big chunks of your data, and now you have update problems because 1) you have to hunt all that down and 2) you don't have transactions that span collections. Also the aggregation language is tedious compared to SQL, requiring you to do most of the work of a query planner yourself.

* Some other person in this thread said MongoDB was faster than Postgres, but I found quite the opposite to be true. For the same real-world workload, basic aggregations on an index, we found Postgres to be much faster than Mongo. No idea what that other person is talking about.

<comment><USER4> Very well put... and this _was_ the point I was making about "decent" performance. If you have super-special requirements (you don't), you'll probably discover it along the way to SUCCESS. If you don't any old SQL database will probably be more than sufficient _AND_ it will be flexible enough to allow you to evolve your schema along the way.

</comment></comment></comment></comment></comment></comment><comment><USER6> This is a huge concern for me at my current organization. Dev has decided to put all data into mongoDB. Yet all decisions are based on that data and the tools we have do not allow for seamless flow (ETL) from mongoDB. That data is important for deriving decisions that affect revenue and costs. Where are solutions for the data analysts and scientists? Frankly I'm pretty sick of hearing it can just be automated.

In my mind there has to be a decent "business intelligence stack". I'm not sure I'm coining that because I didn't get good search results from that phrase. Believe me I've been trying to find solutions. I believe there is big opportunity in building out this sort of stack that bridges data management and data analysis. Sure you can call IBM, Microsoft, Dell, HP but be prepared for big costs and huge software bloat. I would like simplified solutions and options that can fit with most industry standard tools.

I'm also willing to work with anyone on this as well.

<comment><USER7> Others are sharing out of the box solutions.

But I will say that many moons ago when I did actually write stuff for Mongo. The oplog was a god send. You can "tail" the oplog, and get every transaction in near real time. We used this for updating Elasticsearch indexes etc in what is basically realtime, without having to poll or modify existing code at all.

<comment><USER8> The oplog is awesome! It provides an immutable record which is really useful - we materialize the oplog directly in Athena to get a time-travelling database for debugging purposes.

<comment><USER9> Would you mind sharing the process? I'm curious which flow you use to materialize the oplog in Athena/S3.

</comment></comment></comment></comment><comment><USER10> I'm not aware of any analytics platform that runs directly from the source data. There is just about always some kind of ETL process, or at the very least, a data transformation process to shape the data as needed, to provide data that works well for the reporting. So while information on making MongoDB performant for such things is mildly interesting... it just isn't how analytics are generally architected.

</comment><comment><USER11> What is the benefit of having it in mongo in the first place, in this scenario?

<comment><USER12> The people who write the business logic and the people who do the analytics have different concerns. It's sometimes better to make different database choices for those two systems and just copy the data into the analytics system, rather than make a substandard choice of database to try to accommodate both.

If the devs want to use Mongo, it's their problem -- it shouldn't matter much to the analytics people, because they can just copy the data into a different database that fits their needs.

<comment><USER13> Except that devs have to do ETL every day so analysts can do their query work.

<comment><USER12> That can be automated.

<comment><USER13> In theory, yes; in practice, not really.

<comment><USER14> I tend to disagree. Having multiple automated ETL processes running for different projects/clients/colleagues I see that the code does not change as often, as I had anticipated.

Automation here (in my case) is a net win on time.

</comment></comment></comment></comment></comment></comment><comment><USER15> I kind of a hoped it'll end up a joke saying "Don't use mongo". Last time I used it was 2.4 and it was the worst db experience ever. Back then It was more sane to craft a solution with PG and HSTORE. Now, I think RedShift does the job, why would anyone use mongo on production for anything today?

<comment><USER16> It's not too far from that joke.

It's like if you ask "how do I drive my car downtown" and I answer, "Easy, just park at the station and take the train".

To answer your other question, their marketing goes a long way. I recently started at a new company, and the lead was proudly telling me how the project was developed using Mongo... So I start explaining how it's basically shit after using it professionally for a few years. His answer? But SQL doesn't scale well enough!

<comment><USER5> Why is it basically shit? It appears to store and retrieve the data as per my instructions.

<comment><USER16> Except when it doesn't. We've had data corruption issues related to oplog, out of sync secondaries and excessive resource usage on the primary. As far as major problems. There were also a bunch of smaller problems but in fairness those were on the nodejs/mongoose side of things. Would not recommend.

</comment></comment></comment></comment><comment><USER17> Just try this out: [https://github.com/EXASOL/docker- db](https://github.com/EXASOL/docker-db) and you will be impressed. This is an embryo of a real analytical database.

Pros:

\- an 8 CPU installation with 64gb memory will probably be hundred times faster then postgres.

-it supports full sql

\- It is super stable, even as docker container

Cons:

\- it does not support nested data

\- once you reach volumes of around 2Tb, you will probably have to switch to a paid version (I mean, you still can continue running on a 200gb ram box, but it will be suboptimal)

P.s. I am not affiliated with Exasol.

<comment><USER18> > an 8 CPU installation with 64gb memory will probably be hundred times faster > then postgres.

"Probably" not.

The way this usually goes down is that there may be a few synthetic benchmarks show a large performance benefit over existing established databases (x2, not x100), with any non-synthetic benchmark showing very poor performance (1/10th, 1/100th, sometimes even worse), and also often very unstable performance.

The product is then also usually beta quality, as it is hard to compete with the 36 years Postgres has been in development since its inception in 1982 (and that's not counting the 9 years of Ingres development, which Postgres—"Post- Ingres"—spawned from). Important features are usually also quite lacking.

If someone claims x10 or x100 performance improvement over established databases, they better have published a few papers about all the computer science research they must necessarily have done to get there.

<comment><USER19> Full disclosure - I currently work for Exasol.. but I thought I'd just clarify that Exasol has been around for over 15 years and is far from 'beta' (currently on version 6 with hundreds of production installations worldwide). I've also been in the industry for > 40 years and worked with many database products (including Ingres and Postgres) - and all I can say is download the free community edition from the Exasol website or the Docker image as described above and try it for yourself - you will be up and running very quickly and I think you will be pleasantly surprised regarding both functionality and performance.

<comment><USER18> My comment was more general in the sense that such a grand performance statement needs some serious backing, and new products claiming to be several orders of magnitude faster than established products are _usually_ unable to deliver anything at all.

Would you mind sharing some of the differences to, say, Postgres, and what to expect if moving from Postgres to Exasol? Porting my applications to Exasol to benchmark would be time consuming (synthetic benchmarks are very uninteresting), and without any information about what to expect, it simply wouldn't be sensible.

I tried to look at the website, but I am not interested in accepting a privacy policy just to get a white-paper, which frankly leaves me with no usable information at all. The rest of the website is basically empty, short of graphs without data and marketing "You want to do X? We can do that too! <no additional info>". The only real thing I could extract was "in-memory database".

To me, "in-memory database" would appear to be the catch that makes it an entirely different product than Postgres, catering to an entirely different payload with different pros and cons, rather than an faster all-round product. None of my tables fit in RAM anyway.

</comment></comment></comment></comment><comment><USER20> There are several companies, including mine (Fivetran) that will replicate MongoDB into a columnar data warehouse for analytics. For most people, a commercial replication tool + a commercial columnar data warehouse is the best trade off of cost/ease of use. Commercial DWHs deal with all the details of patching columnar formats under-the-hood, and commercial replication tools like us will deal with all the complexity of things like the mongo oplog. For not that much $ you can have a working system in like a day.

</comment><comment><USER21> Okay, we get it, Mongo sucks. Or at least that seems to be the consensus. From what I can tell it seems they've improved their tech _a lot_ though, and I have to wonder if a lot of the "mongo sucks" sentiment comes from either 1. Using early versions of Mongo that really did suck or 2. people having used Mongo at companies where nobody really knew how to use Mongo that well.

</comment><comment><USER22> Dremio helps with a lot of this, particularly the speed aspect – uses Parquet as well as Apache Arrow. (I work at Dremio.) Speeding things up: [https://docs.dremio.com/acceleration/reflections.html](https://docs.dremio.com/acceleration/reflections.html)

<comment> nevi-me Dremio quickly becomes useless with MongoDB given that for a while it's not been possible to join data from two MongoDB collections by their object IDs. Last time I checked, Dremio mangled the id into some string that can't even be matched to the same id on a separate collection.

I had data in PG and Mongo, but couldn't join it together. I asked about this on the forum, was told it's a known issue; and it seemed to end there.

I resorted to doing my analytics by hand in the end, MongoDB's aggregation framework is good enough. Create views from aggregation queries, and it becomes easier

The downside is that one needs a business license to use the BI connector.

<comment><USER23> > The downside is that one needs a business license to use the BI connector.

Have you looked the postgres mongo fdw[0] before?

[0]: [https://github.com/EnterpriseDB/mongo_fdw](https://github.com/EnterpriseDB/mongo_fdw)

</comment></comment></comment><comment><USER24> Okay so... To make MongoDB not suck for analytics, ETL it in a different format. For engineers trained in backed systems, this is pretty obvious. After reading this, I also don't know why I'd choose Pequot things over any other thing.

Baby's first ETL -- just scan the db with a cursor and analyze the data in a script -- tends to cover 90% of the use cases for BI db analytics with almost zero resource consumption anyway. Point being don't write a query to do analytics if your db can't answer your questions performantly, and don't build [latent, stale, slow] Enterprise ETL unless you really need it.

<comment><USER25> As someone who grew up around the home if the Pequot tribe, I'm amused by the choice made here by your input device's autocorrect feature.

</comment></comment><comment><USER26> I see that most of the `don't use mongodb for analytics` are being down-voted, however I tend to agree with them. For all the people out there looking for the database for analytics please check Clickhouse from Yandex, it's easy to get started, amazingly fast and open source.

Disclaimer: I am not affiliated with Yandex in anyway, just a happy customer

</comment><comment><USER27> We use a similar technique at Interana. Our DB is a column store, but we break things up over the time dimension to keep file sizes of individual columns reasonable. One of these time buckets is essentially analogous to a single parquet file. In addition we split/sort these buckets into smaller buckets as more events are added.

</comment><comment><USER28> This is called ETL, to a data warehouse.

Regardless of the choice of primary database, this is nothing new and just shows how a lot of startup technical talent seems to be discovering the same things all the time, usually with needlessly convoluted approaches, and writing blog posts about it.

</comment><comment><USER29> Little bit offtopic but what product did you use to create those visualizations?

</comment><comment><USER30> For those seeking tl;dr: The answer is not to use MongoDB.

<comment><USER12> That doesn't get you out of having to face the problem. This is not a challenge unique to MongoDB or other NoSQL databases. Oracle or Postgres might be ideal for your transactional data store, and a columnar database might be ideal for your analytics.

I suppose you could choose one of those options and sacrifice either your customer experience or your analytics, but it's probably better to use the best database for each use case.

</comment></comment><comment><USER31> > How to make MongoDB not suck for analytics

Easy: you don't use Mongo

</comment><comment><USER32> Protip: MongoDB works absolutely best for analytics when it is replaced with a sane and scaleable column-oriented database like Redshift or BigQuery right before serving that report.</comment>


How Welfare Reform Ruined Public Assistance for the Very Poor - <AUTHOR> http://www.theatlantic.com/business/archive/2015/09/welfare-reform-americas-poorest/403960/?single_page=true <comment><USER2> The headline actually is, "America's Poorest Are Getting Virtually No Assistance"

<comment><AUTHOR> Ah - the other title was from the automatic bookmarking thing</comment></comment>


Apple took my laptop :) - <AUTHOR> http://shubhamgoel.info/?p=249

<comment><USER2> I had the same thing in January. My Macbook Pro 15'' early 2008 was replaced for free with a MacBook Pro 15'' mid-2010. Nice thing is that when I bought the Macbook Pro I took the matte display (I think it came for free). But now the matte screen costs some 150$ more than normal glossy display..

However, since I had the matte in my old one, the gave me the matte in the new one..which come with 1680x1050 :D

<comment><AUTHOR> congrats. I heard some people have also been able to squeeze out upgrades like MBP for a white macbook and a 17" for a 15" MBP. But in all those cases there was a lot of shouting involved.. something that I would avoid.</comment></comment>


Ask HN: A web magazine about song writing - <AUTHOR>

I had an idea about a "web magazine" about song writing. I would interview songwriters about how they work, what they thought while composing a song, why they arranged it that way... a little bit like http://www.youtube.com/watch?v=ci-yIVsDIdk but in text.
I would take snippets of a song and have it decorticated by the artist.
To monetize, I would sell 3$ or 5$/month to people to have access to the interviews before everyone else. Maybe put ads but not sure about this... <comment><USER2> It sounds like a nice hobby, but I'm not sure it's a sound business.

Your content is going to have to be pretty stellar to be able to charge $3-5/month, and you're going to have to have a fairly significant subscription base if you want to cover your expenses and pay yourself (and your writers, etc.) a decent wage.

And that's assuming, of course, you a) have easy access to some top-tier songwriters, and b) are a great interviewer. (Many great songwriters are notoriously difficult interviews....)</comment>


Ask HN: Why do many startups use app.*.com - <AUTHOR> It seems like an increasing number of startups are using app.startup.com as the domain for their main app - but I seldom see this with larger companies.<p>Is there a particular reason? <comment><USER2> To separate the marketing website (www.) with the webapp (app.).

It allows you to scale one independently of the other, and you can have different codebase and deployment workflows.</comment>