Safari adopts the Chrome/Firefox extension model - <AUTHOR> https://developer.apple.com/news/?id=kuswih5l <comment><USER2> I think this was only a matter of time. The Chrome extension model has become the de facto standard, has a large existing extension ecosystem and huge user base, and other browsers have adopted this model. While sticking to their own extension model would ensure that they remained in control, this is the right thing for users, both current and possible switchers. Here's why:

\- Current Safari users can get the goodness of Chrome extensions, specifically the ones focused on privacy such as NoScript and uBlock.

\- Switchers from Chrome can feel familiar with Safari if their Chrome extensions are available to them in the new browser.

\- New macOS users don't need to select a browser based on whether extensions are available for it or not.

With extension availability out of the way, the discussion on macOS browsers is going to come back to UX, speed, and battery usage. UX is a personal choice. Based on my experience, Safari is the fastest and least battery hungry browser on macOS. Syncing using iCloud Just Works (TM) for my devices and account. YMMV.

Net-net, kudos to the team for getting past their "Not Invented Here" instincts and going with the market flow.

<comment><USER3> NoScript and uBlock Origin will not be happening for Safari. The API is missing and won't be implemented. Safari does not have the same full API as Chrome or Firefox. Instead, Safari relies on a more limited, Apple-specific content blocking API.

<comment><USER4> Isn't it along the lines of what Chrome's been talking about moving to, where you basically get to tell the browser what to block but not get at the requests in the way you can now?

<comment><USER5> Pretty much.

</comment></comment></comment></comment><comment><USER6> > BlockingResponse not supported.

> Blocking requests not supported.

so uBlock Origin is not going to happen.

[https://developer.apple.com/documentation/safariservices/saf...](https://developer.apple.com/documentation/safariservices/safari_web_extensions/assessing_your_safari_web_extension_s_browser_compatibility)

[https://www.reddit.com/r/uBlockOrigin/comments/hdz0bo/will_u...](https://www.reddit.com/r/uBlockOrigin/comments/hdz0bo/will_ublock_origin_back_to_macos_big_sur/fvoc7wk/)

<comment><USER7> Im using "AdGuard For Safari". Its open source, in Mac App Store and works well enough to keep using Safari. Trick is to disable two out of 8 extensions that require full page access, leaving on only passive, content blocking ones.

<comment><USER8> Good deal here, no affiliation/affiliate link: [https://stacksocial.com/sales/adguard-family-plan- lifetime-s...](https://stacksocial.com/sales/adguard-family-plan-lifetime- subscription)

Best purchase I've made in a long time.

<comment><USER7> This is different thing to one i posted above, just a note.

</comment></comment></comment></comment><comment><USER5> To summarize a couple of points about Safari's extension functionality for those that are not familiar with it:

* Safari used to have a "legacy" (and proprietary) web extension API that was essentially a bundle of HTML/CSS/JavaScript and was (usually) distributed through the Safari Extensions Gallery. Distributing extensions this way was free. While never a huge platform due to Mac Safari's relatively low market share, it had a number of fairly decent extensions.

* In OS X El Capitan Apple rolled their extensions signing program into the Apple developer program, which costs $100/year. Extensions that were not signed would be "untrusted" and would be quite annoying to install. Many extension developers openly criticized the move–some refused to continue supporting Safari.

* Almost simultaneously, Apple introduced their content blocking API, where extensions could declare a list of what are essentially regular expressions to block content. This model is quite similar to Chrome's declarativeWebRequest API in that extensions have little flexibility at runtime to vet requests. This API works in Safari on macOS as well as 64-bit iOS devices.

* With Safari 12 Apple deprecated the old web extensions model (no longer loading these) and introduced a new extensions model, App Extensions. These are native bundles that run out-of-process and communicate with JavaScript that you can inject into the page. Again, these bundles must be signed with a (paid) developer ID certificate and be sandboxed. Distribution via the App Store is recommended but not required. Any remaining extensions that share code across browsers (i.e. not using native code) are now locked out of the platform.

* (This announcement, which is a rehash of the one at WWDC from this June) In Safari 14 Apple is supporting a conversion tool that converts your standard WebExtension into something that Safari can load, with the same signing and distribution restrictions as app extensions (costs money, sandbox). Many simple APIs are available, but there is the notable absence of the ability to block requests–so uBlock cannot be ported, for example. NoScript does not seem possible to implement either.

* Safari on iOS has never supported extensions–this hasn't changed. The only extensibility available there is still just content blockers using declarative lists.

<comment><USER9> Is $100 a year that big of a deal?

It also blocks out anonymous developers.

<comment><USER10> It's the perfect storm of being very affordable for scammers as a cost of business but being attached to a fundamental lack of anonymity and a single point of control that makes it so dangerous for anyone developing tools to fight oppressive governments etc.

</comment></comment></comment><comment><USER11> This is really exciting for me! Apple reached out earlier this year to ask if they can use my own extension to test the conversion tool (porting from the Chrome version) and to demo it during WWDC. Of course, now I have to convert it myself for Safari 14 and distribute, but it seems straightforward. I hope RecipeFilter can expand to the major mobile browsers one day as well...

Here's a clip from WWDC where they demo the ported extension: [https://www.youtube.com/watch?v=Kwh2y6VkzoA](https://www.youtube.com/watch?v=Kwh2y6VkzoA)

Chrome Web Store for it: [https://chrome.google.com/webstore/detail/recipe- filter/ahlc...](https://chrome.google.com/webstore/detail/recipe- filter/ahlcdjbkdaegmljnnncfnhiioiadakae)

Source code: [https://github.com/sean- public/RecipeFilter](https://github.com/sean-public/RecipeFilter)

The original HN comment that caused me to create the extension almost 3 years ago! [https://news.ycombinator.com/item?id=15745914](https://news.ycombinator.com/item?id=15745914)

<comment><USER5> Cool! Did they have you sign a bunch of NDAs; did they end up working with you at all?

<comment><USER11> I did, yes. It was maybe a month in advance of WWDC and I wasn't able to talk about the coming changes to Safari with anyone.

The couple of biz dev/relations/acquisitions people I met via email and video calls were very nice, but they didn't give me much at all in return for releasing permission for the recording and a quick-turnaround change to the extension to fix a couple things.

</comment></comment></comment><comment><USER12> It's going to be an interesting war to follow, but not to be in the middle of. Chrome is already nuking the current state of things by introducing sweeping changes in Manifest V3. The one that caught the mainstream was webRequests and breaking content blockers. What's not talked about is how moving the "background" process from a long lived hidden web page to a short lived, event based service worker, breaks the actually useful cases of extensions.

For our RSS extension it breaks a bunch of things, like polling for updates in feeds, removes any sort of complex persistent storage like IndexedDB, the ability to open a websocket connection for a realtime connection to backend infrastructure (if the user chooses to use our cloud syncing).

Now Chrome has also made a sweeping change by hiding all extensions you install by default. Hiding away the main interface element of the extension. This breaks the "unread" count or the point of the browser action. How many users will actually understand that they need to "pin" extensions to the toolbar? How many extensions will you have installed that you forgot to remove?

It's like the 90s again. All other browsers have moved to Chrome's extension system, which by no means is a standard. Firefox has explicitly said they will support content blockers in their current form, but not their stance on Manifest V3. Safari has disabled a large portion of the API's, and being just released, what's their story for MV3? Since there is no standard there is no work to keep the API's in sync. Firefox throws exceptions on parameters that Chrome supports. It's really demoralizing.

<comment><USER13> how long will it take chrome team to introduce manifest v3 and drop support for v2? I hope they give sufficient time to devs for migration.

<comment><USER12> Nobody knows. Their stated goal was release in 2020 with 1 year deprecation for MV2. Corona might have changed this. Their is no good source of information and the Chrome extension devrel team is woefully understaffed.

</comment></comment></comment><comment><USER14> Announced back at WWDC: [https://techcrunch.com/2020/06/25/apple-will-let-you- port-go...](https://techcrunch.com/2020/06/25/apple-will-let-you-port-google- chrome-extensions-to-safari/)

</comment><comment><USER15> No uBlock, no thanks. Any battery saving improvements given by Safari is probably negated by the volume of ads getting thrown down my throat.

<comment><USER16> Just curious, what sites aren't taken care of by a decent Safari content blocker extension? I've been using content blockers on both macOS and iOS for years now, and for 99% of sites I visit look barely different from FF/Chrome w/uBO. If I run into weirdness with sites loading it's usually due to something like the site doing something funky like piping content through a web worker.

<comment><USER17> For me, Safari content blockers work very well except for a single website: mobile Reddit. Blocking the Reddit app promotion banner disables scrolling, and Safari content blockers aren't flexible enough to prevent this AFAIK.

</comment></comment></comment><comment><USER18> If they are distributing via the App Store, I assume this means requiring the publisher to be paying the yearly $99 developer fee?

<comment><USER19> Yes, and that is actually already the case. When Apple finished deprecating .safariextz extensions in favor of Safari App Extensions, they also killed the Safari Extensions Gallery in favor of distribution through the App Store. And as you assumed, the developer does have to pay the annual fee.

See [https://lapcatsoftware.com/articles/decimation.html](https://lapcatsoftware.com/articles/decimation.html)

<comment><USER20> This is just ridiculous on the one hand it consolidates their existing tech. On the other hand it gives them too much control over web extensions as a result and they dont seem to have the nicest rep running the iOS app store.

</comment></comment></comment><comment><USER21> I've unfairly disregarded Safari on MacOS for years because it didn't have parity with chrome/FF in terms of extensions. There's uBlock of course, but I spend a lot of time on Reddit, and I can't bear it without the RES extension. So now there's hope.

<comment><USER22> > I've unfairly disregarded Safari on MacOS for years because it didn't have > parity with chrome/FF in terms of extensions.

That seems like a very fair reason to disregard a browser, imho.

</comment></comment><comment><USER23> IMHO, Safari is a restrictive browser. Even if they start using chrome models, their approval process and permission process can be a nightmare and may not be actually something that small/free extension makers should be doing.

</comment><comment><USER24> To my knowledge, Safari already had Chrome extensions working fine (before some conversion to native programs apparently). The problem wasn't in the extension support or even in the 100 bucks fee, but in their extension store's ineptitude in publishing whatever developers were trying to ship. 'Reddit Enhancement Suite' (iirc) stopped supporting Safari for that reason.

Edit: not RES, they cite the move to native code as their point of divergence. But I remember reading somewhere about Safari store's unenthusiastic publishing of extensions.

</comment><comment><USER25> Does this mean I can have them on an iPhone one day too?

A true NoScript alone would be awesome.

<comment><USER26> I'm dying to write an iOS Safari extension that will skip over Google AMP pages to real web.

<comment><USER27> This is what drove me to switch to duck duck go

<comment><USER26> Yeah, I've been on DDG for a couple years now. It has not grown on me.

</comment></comment></comment></comment><comment><USER28> Safari versions can never work on older Mac OSes, correct? I would love to stay on High Sierra or Mojave and get Safari 14. I'm assuming since it is so integrated into the OS, it uses Big Sur specific APIs and references.

Too bad. I am happy enough with any macOS from a couple of years ago, but this Safari change is the first thing I can readily recall caring about for an upgrade.

A lot of my reasoning is liking Safari being lightweight. Upgrading to Big Sur would probably far more than cancel out any gains of using Safari more.

Safari still doesn't appear to have profiles or an easy way to hack it. Good Safari web extensions that can help toward that might not come out that quickly if the extensions need some rework and packaging and be published with the $100/year fee.

<comment><USER29> Updates of Safari are made available for the two preceding releases of macOS. For example, Safari 13 shipped with Catalina, but it's also available for Mojave and High Sierra.

Any OS features not available on those earlier OSes are disabled, of course, but Web features tend to be unaffected. For example, Safari 13 doesn't support dark mode on High Sierra.

<comment><USER28> Oh awesome. I never noticed or thought about it before. Seems like this means web extensions will be available on Mojave. Looks like that'll be my OS until I upgrade computers.

Thanks.

</comment></comment></comment><comment><USER30> But not for IOS?

They want you have to have an Apple developer account, even though the extension is cross-platform. We need a separate "WebExtensions Store" that's cross-browser.

</comment><comment><USER31> It seems feasible within a reasonable amount of time that the browser could have everything that an app would have, or at least close. But to me for reasons of profit, not user experience, companies like apple choose not to make this possible. I'm sure there's always a reasonable explanation of why I'm wrong. But I'd love to hear why.

</comment><comment><USER32> Really hoping this means I can use uBlock Origin again, the adblocker selection for Safari is currently extremely poor.

<comment><USER5> Sadly not at the moment :(

</comment></comment><comment><USER33> I wonder when will webkit devs face their mediocrity and the reality of their Haiti level human resources poorness and make the not dumb move of switching to improving chromium just like microsoft do. It's only a matter of time before major webapps stops working on webkit

</comment><comment><USER34> The most interesting thing here is, that this could bring support for WebExtensions to WebKitGtk! Therefore to Epiphany, which really need Extensions. Interestingly enough, I have the feeling leaving Gecko as foundation was a wise decision years go.

And other webbrowsers like Midoria, Otter, Qute.

</comment><comment><USER35> my god. even for something simple you need xcode and join the apple developer program. $$$ for sure.

</comment><comment><USER36> This is pretty useless if developers have to pay $99/year and a Mac to publish. Most extensions still won't appear on the Safari store.

<comment><USER37> just realized i've had a mac since 2017 and only visited the safari store once

</comment></comment><comment><USER38> Safari _still_ can't use uBlock, so I _still_ can't use Safari.

The content blockers on Safari on iOS are pretty much worthless. Mobile web on iOS is easily the worst part of the iPhone experience, but at least on MacOS there are other browsers. I can't fathom why Apple cripples Safari this way on the desktop. It's at a huge disadvantage, and remains so. Literally the only advantage for Safari in my experience is that it can actually play full-HD Netflix streams, where other browsers can't.

<comment><USER39> > _I can 't fathom why Apple cripples Safari this way on the desktop_

Privacy. For every person installing uBlock Origin, there are going to be 10 people install SuperBlock WowBlocker which is actually just harvesting all the URLs they visit and sending them to some data farm.

<comment><USER38> Saving me from myself is not the greatest strategy when all it accomplishes is to make me use a different browser that avoids this type of nanny treatment.

</comment></comment></comment><comment><USER40> Meanwhile Firefox alienates its current users by killing extensions and publishing an unfinished product.

</comment><comment><USER41> But only on macOS. That literally would be the one change that would make me adopt iOS, you know. I NEED userscripts.

</comment><comment><USER42> Waiting for moderators to change the post title to the original but less descriptive "Easily create web extensions for Safari."

<comment><USER5> Hacker News does not like title editorialization, but it does not like information-light marketing titles either. I suspect it might stick around.

</comment></comment><comment><USER43> To develop a large extension consider using webext-redux[0] and react- extension-boilerplate [1]

[0] [https://github.com/tshaddix/webext- redux](https://github.com/tshaddix/webext-redux)

[1][https://github.com/kryptokinght/react-extension- boilerplate](https://github.com/kryptokinght/react-extension-boilerplate).

<comment><USER44> Webext redux is one of my favorite libraries. I've built some great stuff with it (and its predecessor).

One use case is to create a single page app feel on websites that aren't single page. We use it in Salesforce, specifically.

<comment><USER43> Can you share some solutions to pain points you encountered developing an extension?

<comment><USER44> There's a lot that goes into it. Can you help guide the question a bit more and I'll be happy to give it a go?</comment></comment></comment></comment>


Vandalized Xiaomi Store in Hong Kong - <AUTHOR> https://video.twimg.com/ext_tw_video/1185930141513351169/pu/vid/640x352/U4S2OsdnND9NLCCW.mp4 <comment><USER2> it looks like an intense fire right at the door. the security gate has actually burned and melted. the room beyond the door seems to be uncharred.

Can anyone translate the slogans tagged on the wall?

<comment><AUTHOR> On my own I could only translate the word on the left, "狗官". The first character means "dog", the second one means "officer" (that one is the same as in "警官", which means "police officer").

</comment></comment><comment><USER3> Source?

<comment><AUTHOR> SCMP, it's a pro-PRC publication but the footage seems to check out. The store pictured is in Mong Kok, this area has seen major unrest today. [https://twitter.com/SCMPHongKong/status/1185930191039655938](https://twitter.com/SCMPHongKong/status/1185930191039655938)

I wouldn't jump to a conclusion as to who was behind this—seems about the type of thing one would do to provoke military intervention.

Protesters do half-heartedly distance themselves from this sort of behaviour, but this instance might warrant a louder gesture from their side. Sadly, they're way too disorganised to bother (or even to be capable of) issuing such a statement over a mainstream channel.

Lack of organisation makes the whole movement accountable for actions of anyone wearing a black shirt, and thus is very easy to undermine, purposefully or accidentally.

</comment></comment><comment><USER4> Vandalized? That's completely destroyed.</comment>


Flux Theory (of everything) - <AUTHOR> http://guitar.to/gravityboy/

<comment><AUTHOR> This was linked in the comments of the 10 dimensions article of New Scientist.

In the article there was also a mention of another of this theories, E8:

[http://en.wikipedia.org/wiki/An_Exceptionally_Simple_Theory_...](http://en.wikipedia.org/wiki/An_Exceptionally_Simple_Theory_of_Everything)</comment>


NFV Platforms with MirageOS Unikernels - <AUTHOR> http://unikernel.org/blog/2016/unikernel-nfv-platform <comment><USER3> I really want to play with HaLVM at some point. Unikernels are fascinating to me. Think of how many server hours we can save by just booting up a unikernel on each incoming request.

The problem with HTTP, for me, is that I need to have some program running, listening on a port, in order to access the functionality that I've chosen to expose behind a REST API.

With unikernels, RESTful web servers go from being a special kind of function, that requires an always-running executable, to the executable being "applied to" incoming requests, as needed. So a DNS server can basically take an incoming request, boot up a unikernel in 20 ms, reply with the unikernel IP, and the newly started unikernel will receive the actual request data shortly thereafter. This feels more like "applying a web function" to an incoming request, and throwing away that function when the connection is closed - <USER2> a garbage collector, of sorts. Rather than having some executable sit and wait for data on a TCP socket, it would be nicer to be able to distribute web apps as shared objects: loaded into memory when needed and applied to a request.

<comment><USER4> This actually sounds like how you would do this in Erlang - and it's really elegant because WebSockets/Server-Sent Events are simply encapsulated in processes spawned when the request is received. I don't know if this is how Erlang webservers actually work, but this is how I'd do it.

<comment><USER5> you might be interested in [http://erlangonxen.org/](http://erlangonxen.org/) (LING) if you haven't seen it already!

<comment><USER6> I'm pretty sure this has been abandoned. Hasn't had any updates in a really long time. "Latest commit bc97a26 on 12 Oct 2015".

One of the contributors recently wrote: maximk commented on 25 May "All recent updates to the code were customer-driven. New users mean new updates. The likely application area for this is NFV, modular software for telecoms. No definite plans"

[https://github.com/cloudozer/ling](https://github.com/cloudozer/ling)

<comment><AUTHOR> It's not abandoned AFAIK, it's a small team that does consulting and they have to prioritise work for customers. When that coincides with Ling they can make progress. I've heard this from more than one unikernel/LibOS project.

<comment><USER6> Well that's better than it being dead in the water. It looks pretty cool.

</comment></comment></comment></comment></comment></comment><comment><USER7> How are most of you able to do uni kernel deployment and development? I've found that my primary deployment target (AWS EC2) is disqualified since most of these run times need hardware virtualization support.

<comment><USER8> I think EC2 supports both PV and HVM guests, right?

<comment><USER7> Yeah, but you can't really run a unikernel directly on EC2, you'd want to run RHEL / Ubuntu / etc.. on EC2, and then a unikernel in that.</comment></comment></comment>


A fast geo database with LevelDB, Go and Geohashes - <AUTHOR> http://blog.nobugware.com/post/2015/leveldb_geohash_golang/ <comment> maze-le What this "geo database" lacks is the ability to make actual spatial queries.

It is one thing to store and retrieve points, lines and polygons in a database. This is what this stack can do quite well and I suppose for a certain class of geospatial applications, this is the only capability needed. But what you can not do is perform complex operations like: Projection/Reprojection, computing polygon-intersection, computing a convex hull of a set of geometries, compute a centroid, compute an area... (sort of everything that postgis can do).

This is not strictly a "geo database", more a key-value store with the ability to store and retrieve geospatial data.

<comment><USER2> I believe it's even more limited to just storing points. One could do a query within a bounding box that does not cross the international date line, and maybe a radius search and that's about it.

Geohashes are good in that it doesn't need any fancy indexing as nearby points are also nearby in the index.

</comment></comment><comment><USER3> Hey guys author here, didn't know it was posted on HN.

You are right about complex geospatial queries, it's not the purpose of this simple storage. But someone sent me a promising lib from Google: [https://docs.google.com/presentation/d/1Hl4KapfAENAOf4gv- pSn...](https://docs.google.com/presentation/d/1Hl4KapfAENAOf4gv- pSngKwvS_jwNVHRPZTTDzXXn6Q/view)

It could be applied to this model to make it handle polygons.

</comment><comment><USER4> A few quick code comments:

Now().UnixNano() is not guaranteed to be monotonically increasing. I'd be more careful if your keys really do need to be unique.

You can use time.Truncate to get the 10 minute interval.

        var t time.Time
        t = start
        for {
        	if t.After(stop) {
        		break
        	}
        	/* ... */
        	t = t.Add(d)
        }

Would usually be written:

        for t := start; !t.After(stop); t = t.Add(d) {

<comment><USER3> It's not actual code but notes for the simplicity of the blog post. Thanks for the advise I didn't know about Truncate.

</comment></comment><comment><USER5> I was really excited for a new geo database, especially one involving Go. But, this is extremely limited. I could see some uses for it, but how does it do when you are storing hundreds of millions of points (or more) rather than a few million?</comment>


Jetbrains announces code review tool Upsource - <AUTHOR> http://www.jetbrains.com/upsource/

<comment><USER2> Has anyone tried it ? I was interested on how does it matches up to Phabricator ?</comment>


Ham Radio Growing In The Age Of Twitter - <AUTHOR> http://www.npr.org/templates/story/story.php?storyId=125586086&ft=1&f=1019 Anyone on HN have a ham radio license? <comment><AUTHOR> Anyone on HN have a ham radio license?</comment>


This Is What Happens When Millions of People Suddenly Get the Internet - <AUTHOR> https://www.buzzfeed.com/sheerafrenkel/fake-news-spreads-trump-around-the-world?utm_term=.vnGo7xDGz#.grGJezvVn <comment><USER2> Surprisingly, this is an excellent article. Buzzfeed has a reputation for clickbait crap like "Six disgusting things your cat does at night" but this is a well-researched, thoughtful work of journalism on a serious topic. The page also loaded like lightning and has perfect layout for mobile. Hmm...maybe time for me to re-evaluate my prejudice.

<comment><USER3> Open-minded comments like this are why HN is my sanctuary in the general terrible attitude of the internet nowadays.

<comment><USER4> Yes, I think much of HN culture is way better than on the rest of the Internet.

But the HN community still struggles to admit their own short commings.

I once started a rather big flame-war here just by telling my opinion about religion and if you say something about how most people here are rich (globally seen) white males and the community opinion reflects that, people go crazy too.</comment></comment></comment>


Hello, my name is GeorgeOrwell, and I am a chatbot - <AUTHOR> http://www.cyberkins.com/GeorgeOrwell.php <comment><USER2> Sorry, I couldn't understand what this was doing.</comment>


How to concentrate? - <AUTHOR>

I'm a professional developer for two years now, going into my third year. What I noticed since week one, and i never had an issue with at school or university, is that I can't really work (i.e. program) for 5 days a week, 8 hours a day.
My maximum productive time is about 4 hours a day. This is more than enough to keep up with my colleagues. If I have to do more work, let's say for a bugfix or a critical feature, I almost immediately get headaches which can last for a week or more.
Obviously I'm very unhappy about it. I used to read books in the evenings, learn stuff, now can't even listen to music some days.
Do you have any advice on this? Colleagues and friends i consulted either can't relate to it, or just in a very general sense ("Some days are though"). I'm serious enough to cancel my software developer career if I can't find a remedy some time soon, even though I love to program.
: I actually keep track of my time with RescueTime, trying to limit my "productive time". <comment><USER2> Best advice I have is the 20/20 rule.. I used to do the same thing. I changed the lighting in my office. Dim light except for the monitor. I use Flux to adjust the brightness depending on the time of day. And every 20 minutes I try to focus on something at least 20 feet away. It helps keep eyestrain at bay which is what causes most headaches. Also swap tea for coffee, water for cokes, and I use apps like focus to stop distractions.</comment>


Oovoo annoys the user with unwanted noisy advertisement - <AUTHOR> http://user.wordpress.com/2012/03/22/oovoo-annoys-the-user-with-unwanted-noisy-advertisement/

<comment><AUTHOR> This has been an ongoing issue since 2011 and they still haven't fixed it!</comment>


Laurence Tratt: How can C Programs be so Reliable? - <AUTHOR> http://tratt.net/laurie/tech_articles/articles/how_can_c_programs_be_so_reliable <comment><USER2> First: Pointers do have a simple real-world analogy. They're post office box numbers.

A given post office box is a container with a fixed size and a fixed number. You can refer to a post office box using that number. You can also do things like say "Get me everything from the four post office boxes beginning with post office box 20." or "Go to post office box 3, go down two boxes, and get me what's in the post office box there."

Finally, you can write a post office box number down and stick it into a different post office box. Then, to get anything useful, you have to say things like "Go to post office box 15, open it and read the piece of paper inside, go to the post office box with the same number as the one written on the piece of paper you just read, and get me what's in that box." You can actually 'chain' requests like these for a long way, with one post office box having a piece of paper with the number of the next post office box written on it. This gets screwy after the third or fourth box.

Anyway, the post office is the simplified notion of RAM your OS gives application programs, a given post office box is a RAM location (byte or word), and a post office box number is a pointer.

The rest of the article is basically "You _can_ program carefully enough to make C programs robust." and that is, in fact, true. I use robust C programs every day, one of them being the Linux kernel. However, the actual debate is over whether it's worth taking the time to be that careful when you could use a different language which makes a lot of mistakes impossible.

<comment><USER3> This. I've always wondered what's so difficult about pointers for some people, mainly because I used a similar mental model from the very beginning.</comment></comment>


Sputnik V – The First Registered Covid-19 Vaccine - <AUTHOR> https://sputnikvaccine.com/about-vaccine/ <comment><USER2> We conduct the clinical trials and do the other things not because we are bored, but because using a rushed vaccine we couldn't test for long-term side effects is reckless enough.

<comment><USER3> What would the side effects of the common cold be? A common cold maybe?

<comment><USER2> Is Sputnik just a common cold virus, without any modifications?

<comment><USER3> With mods.

Sputnik was the world's first satellite. Sputnik V (for Vector) is the nickname for the first "approved" Covid-19 vaccine, but isn't really if you look closely. [https://en.wikipedia.org/wiki/Gam-COVID- Vac](https://en.wikipedia.org/wiki/Gam-COVID-Vac)

Vector means that the common cold adenovirus (the base with the big ACE2-attracting head attached but hopefully harmless) is used to infiltrate the body, first as Ad26 variant and then after three weeks the Ad5 variant. This should create enough antibodies in healthy persons to fight off the SARS- Cov-2 variant.

They could have called the Vector also Rocket because it's the carrier, but Sputnik V matches Saturn V much better. Sputnik R doesn't sound that good, and a vector sound less harmful than rockets or carriers.

</comment></comment></comment></comment><comment><USER4> So, this is Phase 3 trials which are due to start on 12th as per the site. The "Clinical Trials" [0] page has some more info. As far as I can see, no published articles in any peer reviewed journals on the efficacy they saw in Phases 1&2.

"Phase 3 clinical trial involving more than 2,000 people in Russia, a number of Middle Eastern (UAE and Saudi Arabia), and Latin American countries (Brazil and Mexico) will start on August 12.....Mass production of the vaccine is expected to start in September 2020"

[0][https://sputnikvaccine.com/about-vaccine/clinical- trials/](https://sputnikvaccine.com/about-vaccine/clinical-trials/)

</comment><comment><USER5> The video intro on this page [https://sputnikvaccine.com/sputnik- moment/](https://sputnikvaccine.com/sputnik-moment/) is made as if it is a conspiracy theory</comment>


Ask HN: Anyone here do Web mining/harvesting/scraping? - <AUTHOR>

I've seen a bunch of software solutions out there. WebQL lets you write SQL-like code to harvest websites. There are also GUI solutions like modenza, fetch and kapow.
Does anyone here work with these kinds of solutions and what do you consider to be the best approach and why?
Thanks! <comment><USER2> I have done some webscraping with twisted and BeautifulSoup. But it was just for a few small sites.

That was when I found <http://scrapy.org/> , a complete python framework for writing webcrawlers. It looks promissing, but I haven't used it so far.

</comment><comment><USER3> We do a lot of scraping and the best we've found, really, is beautifulsoup and being good at python. There's no UI or easy tool that's going to replace that.

<comment><AUTHOR> Actually WebQL is pretty amazing. A simple program like select * from links within <http://www.cnn.com> to an advanced 10 page script to pull and merge multiple sources. The code is real easy to read and if you have any kind of SQL background very easy to learn. We're just always keeping our eyes out for something better/cheaper (ie free) :)

</comment></comment><comment><USER4> I would recommend hosted crawling/scrapping solutions where you don't need to dedicate your own hardware resources. Also monitoring and maintaining the scrappers/harvesters can be a pain. Shameless Plug: I work @ beevolve (www.beevolve.com) and we provide hosted crawling and scrapping solutions. PM me if you are interested.

</comment><comment><USER5> I use a mixture of cURL and PHP's DOM and simpleXML functions to scrape what I need.

</comment><comment><USER6> I've done it with Java, HttpClient and regular expressions.</comment>


Black Tide in Brazil - <AUTHOR> https://correspondent.afp.com/black-tide-brazil <comment><USER2> Total especutlation mode about the origin of this disaster: Venezuela is suffering oil export sanctions. When that kind of sanction happens it's not uncommon to transport oil in ships with "rented" flags or no flags to ports of close countries, so the oil is incorporated to that country's production and can be sold.

It is a tiny quantity, but generate some money to some people. As everything clandestine, this transportation doesn't care much about safety and environmental precautions.

It is tough investigation to make, so maybe we will end only with those especutlations

<comment><USER3> You're right on. Chemical analysis has already confirmed that the oil comes from Venezuelan fields, and the quantity is smaller than you'd expect, they are currently investigating a handful of suspect ships, but most likely the responsible one would have had its comm equipment turned off and hence not recorded by radar.

<comment><USER4> Turning communications equipment off is all that's necessary to disappear from radar?

<comment><USER5> Not exactly...

turning off radios would prevent inadvertent/automatic transmissions

disabling transponders would reduce footprint on more modernized port control radars

not using radar from ownship reduces the chance of being seen

all these together would render the ship's ELINT footprint down to an old school blob on whatever radar plot

that's about as anonymous as you can get without active countermeasures and those are VERY obvious

<comment><USER6> Can you tell us more about the active countermeasures? How do they work and what makes them very obvious?

<comment><USER7> That is "stealth". It's obvious because it is expensive and secret tech. But also because if you look at the ship you see a ship, whereas the radar shows an albatross or something small. Transitions are also notable for radar operators that are really paying attention. A ship that turns into a goose and then back into a ship, for example.

<comment><USER8> I think you may be mixing up active jamming (which works like shining a searchlight in someone's eyes--they can't see or track you, but it's very obvious that there's _someone_ out there) and passive stealth (which tries to reduce the emissions that the vehicle emits or reflects).

Anyone can make a jammer, but it only helps if you can evade your pursuer before they get a look at you optically; it's not much use for a lumbering oil tanker.

As zentiggr says, you can get some degree of passive stealth just by turning off your own radio/radar, but anything more than that requires a custom- engineered (and strangely-shaped) hull, propulsion, etc. that takes military- level money to design and would be extremely obvious in any port. So, again, not practical for this kind of smuggler.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> A bit of local context. The Brazilian president accused Green Peace in staging the accident. Also according to Bolsanaro fish are not dumb and if they see oil they are going to avoid it (that was a real treasure for memes).

<comment><USER10> Can they see the oil? I read mobile fish can avoid it, but shellfish and such can't move that fast if they're already in it.

[https://www.itopf.org/knowledge-resources/documents- guides/e...](https://www.itopf.org/knowledge-resources/documents- guides/environmental-effects/effects-in-offshore-and-coastal-waters/)

<comment><USER11> This is the bullshit asymmetry principle at work:

 _The amount of energy needed to refute bullshit is an order of magnitude bigger than to produce it._

a/k/a Brandolino's Law.

[https://statmodeling.stat.columbia.edu/2019/01/28/bullshit-a...](https://statmodeling.stat.columbia.edu/2019/01/28/bullshit- asymmetry-principle/)

The claim is bullshit. We know it's bullshit. Refuting it ties up resources and conversations.

Flag it as bullshit and move on.

See also burden of proof fallacy:

[https://www.logicallyfallacious.com/tools/lp/Bo/LogicalFalla...](https://www.logicallyfallacious.com/tools/lp/Bo/LogicalFallacies/222/Shifting- of-the-Burden-of-Proof)

</comment></comment></comment><comment><USER12> Ross Kemp did an excellent documentary about the rise of piracy in Nigeria that investigated the motives of the pirates. It actually ended up being about how the rise of massive pollution from oil companies, enabled by government corruption, destroyed the livelihoods of hundreds of thousands of Nigerian villagers who no longer could fish or grow crops.

We can't get off of our oil addiction fast enough.

[https://www.youtube.com/watch?v=XjGl8CLjKh0](https://www.youtube.com/watch?v=XjGl8CLjKh0)

</comment><comment><USER13> There are still some black sand and naturally formed asphalt crust, in Brittany's beaches in France... 40 years after [https://en.wikipedia.org/wiki/Amoco_Cadiz_oil_spill](https://en.wikipedia.org/wiki/Amoco_Cadiz_oil_spill). It is virtually impossible to clean oil spill. Hopefully this one is not too big, and won't spoil too much of the shore.

</comment><comment><USER14> It has been removed 4500 Tons of oil from the northeastern beaches already, according to the Brazilian Navy ([https://www.marinha.mil.br/sites/default/files/nota_gaa_17no...](https://www.marinha.mil.br/sites/default/files/nota_gaa_17nov_.pdf)).

<comment><USER15> Wow. There are roughly ten thousand people involved in the cleanup effort, the majority from the Navy and the Civil Guard. Huge props to these folks.

</comment></comment><comment><USER16> there's no way to just burn it when it dries on the beach?

<comment><USER17> I suspect that without a well-sealed oven, you just get toxic air and soil if you burn it.</comment></comment>


Heavyweight physics prof weighs into climate/energy scrap - <AUTHOR> http://www.theregister.co.uk/2008/06/20/mackay_on_carbon_free_uk/

<comment><USER2> Finally, somebody speaks logically and with credibility about how little actions have no practical effect on energy use or carbon output. Like he said, if you eat hot-cooked and/or refrigerated food, drive to work, and are socially acceptably clean, then frankly you use a ton of electricity. Disconnecting your phone charger does nothing.

Hopefully the mass media will report on this in the US.

I was surprised to learn that current nuclear technologies don't have the throughput to handle a lot of our energy needs in the long-term. Hopefully the new fission technologies (or even better, fusion!) get developed and put into practice.

<comment><USER3> I agree - it's overdue for someone numerate to analyze the options. I've downloaded the book and will read it this weekend. It's always astonished me that most mainstream journalists focus on stuff like not leaving your TV on standby, but never suggest turning the heating down a bit (or turning the A/C off for any of you lucky enough to live somewhere warmer than where I live)

<comment><USER4> a TV on standby can actually use 40W of power. This is really an issue of product design that we should be demanding the manufacturers fix. While 40W may be small compared with the bigger picture, it is still energy that is completely wasted, and if you add up the savings from everyone in USA turning the TV off all the way, the energy adds up, and Americans just got a lot more exercise from getting up from the couch and walking to the TV.

Phone chargers and many other electronic devices are using less than 1W when unused but plugged in. Again, this is an issue that should be solved with better product design.

<comment><USER5> _While 40W may be small compared with the bigger picture, it is still energy that is completely wasted_

Not quite - the "waste" is just emitted as heat, which will often be useful.

<comment><USER6> This is probably balanced out by people in hot places who have to expend extra energy on air conditioning.

</comment></comment></comment></comment></comment><comment><USER7> This guy sounds pretty good.

I took a physics class like this ("Energy and Society") from Philip Taylor back at Case Western Reserve University ("a tradition of excellence that's even longer than our name"). To sum up the lesson of the class in a sentence: One of the most useful skills of the physicist is the ability and inclination to do simple back-of-the-envelope calculations, and to take the results seriously.

</comment><comment><USER8> 5 minute shower * (5 gal/min) * (440 BTU [to heat water 85 degrees]/gal ) = 11000 BTU or 3.223kWh or ~16 hours of 200-watt HDTV

1 gallon of gasoline = 115000 BTU or 33.7kWh or 10 5-minute hot showers

Am I willing to take cold showers (not sure..)?

<comment><USER7> But if it turns out that these are the important numbers, you can _fix_ them to an amazing extent. That's the very essence of the Pareto principle: Focus your efforts on the things that matter.

There are, if necessary, alternatives to the daily 5-minute showers of 5 gal/min. (Some people, as we all know but politely avoid mentioning, are used to the alternatives already. ;) It might be a combination of: fewer showers, cooler showers, shorter showers, better heat-recovery technology for shower water, more efficient shower heads that use less water but provide the same cleaning, improved shower techniques involving better soaps and high-tech sponges, special filters for water reclamation...

The transportation situation, of course, is more expensive and tedious to fix, but the answers aren't _that_ hard: Design cities around public transportation. Live at higher density and/or with smarter zoning that leads to shorter trips. Build _real_ bike lanes so that those of us who bicycle in Boston don't need such expensive insurance policies.

<comment><USER9> >more efficient shower heads that use less water but provide the same cleaning,

I think a lower flow but high pressure might be useful here; perhaps supplement normal water pressure with an electric pump. Another possibility is low flow with a trigger that lets you turn on high pressure/flow when you need it.

<comment><USER10> But then people just hack it so it stays on high flow.

<comment><USER11> Why would you do that - when you have to pay for the water and energy?

</comment></comment></comment></comment></comment><comment><USER12> Very much agree with the order of magnitude argument. I never understood all this economic lighting fad: i use economical light bulbs because they save me money, not because they save the planet. Everything I save this way is probably less then one brake of a freight train.

What irritates me more is how much we allow people to push their irrational arguments over us and let them dominate the discussion. I strongly doubt Greenpeace hasn't made the same math, they just don't care. And still they can make someone be afraid to be labeled as pro-nuclear. Why? Why do we allow obvious flawed theories be pushed around until they become common sense?

</comment><comment><USER13> The book can be downloaded here: <http://www.withouthotair.com/>

</comment><comment><USER14> > He says he's largely letting his machine-learning lab at Cambridge run > itself these days

I hope the humor was intentional -- especially since The Register has a whole section dedicated to the "Rise of the Machines™."

</comment><comment><USER15> I find it useful to think in terms of watts, like Saul Griffith does here: <http://www.wattzon.org/>

It puts 1W standby energy in context of a total "wattage" we each use.

</comment><comment><USER16> Very fascinating. We sometimes forget that far from being 'clean' energy, the sun is actually a massive nuclear furnace. It just happens to be far enough away :-)

<comment><USER17> Now I'm going to have that TMBG song stuck on my head for the rest of the day, thanks.</comment></comment>


GCHQ Slides Cribbed from Bell and Whaley's Book 'Cheating and Deception' - <AUTHOR> http://ethanheilman.tumblr.com/post//gchq-deception-slides-cribbed-from-bell-and-whaleys

<comment><USER2> Thanks for the info!

We should remember that UK intelligence has a venerable history of plagiarising from public sources:

> The British government's latest report on Iraq's non-compliance with weapons > inspections, which claims to draw on "intelligence material", has been > revealed as a wholesale plagiarism of three old and publicly-available > articles, one of them by a graduate student in California. The compiler did > not even clean up the typos or standardize the spelling.

[http://www.fas.org/irp/news/2003/02/uk020603.html](http://www.fas.org/irp/news/2003/02/uk020603.html)</comment>


Crossing the Canyon: Product Manager to Product Leader - <AUTHOR> https://www.reforge.com/blog/crossing-the-canyon-product-manager-to-product-leader <comment><USER2> In my opinion, there are many valuable lessons for any IC moving into a people management role. PMs are not as unique as they might think.</comment>


Comparing 3 open source databases: PostgreSQL, MariaDB, and SQLite - <AUTHOR> https://opensource.com/article/19/1/open-source-databases <comment><USER2> > While its community support is very good, PostgreSQL's core documentation > could be improved.

hm... I'm thankful for PostgreSQL's high quality documentation.</comment>


PopSQL – Modern, collaborative SQL editor for your team - <AUTHOR> https://popsql.io/ <comment><USER2> Other solutions in the `write sql and share the graphs with your team`-space:

\- ChartIO

\- 'WagonHQ, Modern SQL Editor' [1] (now aquired by Box)

\- MetaBase [2,3]

\- Redash [4]

MetaBase is the only one that I know of that is

\- fully open source,

\- mature (graphing options, permission model),

\- still actively maintained, and

\- both friendly to non-technical users and expert sql'ers alike

[1] [https://news.ycombinator.com/item?id=9792464](https://news.ycombinator.com/item?id=9792464)

[2] [http://www.metabase.com](http://www.metabase.com) [3] [https://news.ycombinator.com/item?id=10425959](https://news.ycombinator.com/item?id=10425959)

[4] [https://redash.io](https://redash.io)

<comment><USER3> Redash creator here.

Just to make sure someone doesn't get the wrong impressions from your comment: Redash is 99% open source (and I'm going to close this gap this month[1]), mature and actively maintained. The friendliness is subjective, but we're not trying to please everyone :-)

[1] Sometimes it's easier to prototype new things in the SaaS version, but everything reaches open source eventually. There is practically one feature that wasn't open sourced until now, and I'm going to add it to the open source version now.

<comment><USER4> is the feature joining multiple data sources? i'm a fan of redash btw, people should check it out. i use it as a docker container for self hosting

<comment><USER3> It is :-)

</comment></comment></comment></comment><comment><USER5> There doesn't seem to be any PopSQL users around here so I'll share my experience with it.

Currently, we're a team of 5 on a paid plan and we're loving the tool. We mainly use it to:

\- Share queries to extract some kind of data from our databases \- Quickly run queries to answer a quick question \- My favorite use case: We create queries around bugs we've noticed in our data. We add "TODO:" in front of the query's name. We'll then move the query to a "Done" folder once we get the expected result.

</comment><comment><USER6> Or, we could just go all meta, in a Postgres shell:

      CREATE TABLE queries IF NOT EXISTS (
        name varchar(64),
        version varchar(64),
        query text,
        description text
      );

      NOTIFY chat 'Alice: Bob, please insert that query into the new queries table, and then NOTIFY "chat" with the name and version of it';
      LISTEN chat;

      Asynchronous notification "chat" with payload "Bob: See update-rank, v. borked-1" received from server process with PID 8448.
      Asynchronous notification "chat" with payload "Bob: I have it set to update in a temp table, so we don't have to reset the real table" received from server process with PID 8448.

      SELECT queries.query FROM queries WHERE name="update-rank" AND version="borked-1"
      INTO query;

      \echo query

      -- "Looks sane, let's run it"

      DO $$ EXECUTE query INTO result $$; -- Something like that, I'm guessing.

      \echo result

PL/PgSQL's EXECUTE is not to be confused with the EXECUTE that is used with stored procedures.

Bonus points if you edit the query by shelling out to sed or butterflies.

</comment><comment><USER7> Marketing Electron app as native and requiring sign in with Google account only is not an option, just for a try...

</comment><comment><USER8> How are credentials and result sets transfered? Is it directly from my computer to the server or is PopSQL a middle man in the communication?

<comment><USER9> This is a key question! Also want to know as I'm considering signup and data security is a factor.

<comment><USER10> It seems to store the database URL on the server (popsql) but the username and password are stored in the Mac Keychain, any "team members" must enter the database username and password against when attempting to use the connection.

I am unsure of what happens for other OS's.

</comment></comment></comment><comment><USER11> I like the GUI, I like the landing page, I like the fact that it has native versions for the 3 main platforms... Congratulations for the team or person behind this! Great work

</comment><comment><USER12> Question: is the native app actually native or JavaScript in a web frame?

<comment><USER13> Web frame (Electron), at least on Linux.

</comment></comment><comment><USER14> Postgres user here. pgadmin4 lead me to try alternative clients. Recently, I've found DBeaver useful. I'd give popsql a try if it didn't require google account information.

<comment><USER15> I want so badly to like DBeaver. I'm actively using it, but on Linux the rendering always gets screwy with me. Almost every time I start the program I've got to change themes just to get it usable.

</comment></comment><comment><USER16> It's fine to require people to pay for your software. I still think it's a good idea to release at least a basic demo version as open source though. With close binary blobs for all I know it could upload all my personal files or delete them for good, drop all database tables, install a backdoor, etc. I don't trust random people on the Internet to run their code on my computer.

<comment><USER17> This is what permissions are for, we use limited read-only access for all 3rd party tools.

</comment></comment><comment><USER18> Please add Oracle support when you have an opportunity (I was going to give it a quick try, but unfortunately it doesn't appear to have support for Oracle just yet and that's the primary database our internal systems run off of).

</comment><comment><USER19> Being an electron app, can anyone tell me how it behaves when viewing 200, 400, records at a time? I attempted to open PopSQL but I don't really want to sign up for an account just to try the thing out.

I tried out TeamSQL a while back and it choked pretty hard at about 250.

<comment><USER20> Not quite directly SQL, but you can try CSVExplorer.com to work with huge CSV dumps from your db.

(disclaimer: I built CSV Explorer)

</comment></comment><comment><USER21> Just tried this out

\- Chart is greyed out on my data. Why is that?

\- Can I provide parameters (such as a date) in the query that can be changed by whoever wants to run a report? I can't see how to do this.

\- Can I publish this as a dashboard that can be used by non-technical team members?

<comment><USER22> PopSQL co-founder here. Regarding the greyed out chart, we rely on your results being in a specific format to render a chart. Check out this help article for more details on that: [https://intercom.help/popsql/how-to-create- charts](https://intercom.help/popsql/how-to-create-charts). We have plans for advanced charting, query parameterization, and dashboarding!

</comment></comment><comment><USER23> I love the name!

</comment><comment><USER24> >Free for individuals, or teams of less than 3 people

That's a convoluted way to say "1 or 2 person teams"

</comment><comment><USER25> "Stop emailing SQL queries or pasting them in chat"

Hunh you don't you put your queries in sprocs and store that in the data base and your source control system of choice.

Soory I don't see what problem this product is for apart from maybe enabling sub optimal coding practices

<comment><USER26> Story time;

I've recently started working with a SME (<100 users) who took this approach after abandoning the vendor's reporting recommendations for their Line of Business application (apparently it was just crap).

Other areas of the business _cannot_ be without these reports under any circumstances. Negotiation is not an option.

Since about 2012 they've had various employees generating reports, alerts and data warehouses as objects in the database. Naturally some of these have been superseded. Very often the old objects were left behind "just incase". Almost all of the original authors have left and documentation has been lost or just didn't exist in the first place.

Many of these objects are dense and difficult (because SQL was the wrong tool for the job, or because they go many layers deep like matryoshka dolls).

Demands for changes to reports are frequent (weekly), and some reports functionally overlap, but produce wildly different results for different areas of the business based on various "rules".

Honestly, the database is a fucking mess (approximately 1900 objects relating to reports, alerts and data warehouses - some of which are dolls going many many levels deep., There's a huge sense of shame and fear over trying to regain control.

I totaly understand that this was entirely a human problem. With more restraint this wouldn't have happened. Both inside and out side of the IT team. However, a tool like PopSQL, metabase, etc. I believe would've helped with;

1\. Ad-hoc queries that didn't really need to be full objects

2\. Discoverability through prettier annotations, etc. - reduction of reports being duplicated

3\. Auditing - last access/run times, etc. would be much easier to find to reduce the cruft

4\. If the tool is more appropriate it could off load some of the things that SQL is not good at

There are undeniably good reasons for storing reports, alerts and data warehouses (developer maintained objects) in version control. If the report creation/alteration requests don't come often, then I'd argue it's ideal.

However living in the real world, I personally feel it needs to be balanced in conjunction with tools like PopSQL, metabase, etc. Right tool for the right job.

<comment><USER27> I've seen this same story at every job ive ever had or contracted for. Fortune 100's down to mom and pops.

I think metabase/redash/etc is a great fit for the development of reports and the kinds of things that dont need to be anything more than a sql query. IME fast turnaround can be huge for lots of businesses.

I agree with all of your points. A lot of the problem is organizational and political and historical - but the right tool can make a big difference.

<comment><USER26> Any tips on prying people away from Excel, and into something like metabase/Tableau/whatever?

That seems to be one of the biggest hurdles I think I'm going to have to figure out, rather than the technical side.

<comment><USER27> a decade ago I would have had a lot of ideas for you but as time goes on the more I have learned to embrace excel.

Theres two directions you could be coming from though - if its a problem of the users producing data and storing it in excel instead of your database thats more of an application problem.

But if its a matter of excel being where your users want to look at and work with the data - I say embrace that. Use your metabase/tableau/redash to provide them with that data and let them get it out in excel and do whatever they want. Give them a little time and then let them show you what theyre doing with it.

IME about half the time theyre actually doing things with it that you cant really help them with unless you spent a ton of time. The other half you can take what theyre doing and either apply those changes into the query itself, saving them time and sharing that work with others - or teach them a better/easier/faster way of achieving whatever it is theyre after. I use it as a long-tail way of gathering requirements a lot of times - let me give you all the data you might need for this and you figure out how it can be useful for you.</comment></comment></comment></comment></comment>


Layoffs Confirmed At 23andMe  - <AUTHOR> http://www.techcrunch.com/2009/10/29/layoffs-confirmed-at-23andme/ <comment><USER2> 23andme is a concept that needs a lot more groundwork before dumping all that information on the public.

For one it should be accompanied by a basic course in genetics and another in probability theory.

GPs are amongst the most overworked people in any profession and they really don't need to have a crowd of ill-informed consumers pestering them with their newfound diseases.

Personal genetic screening is potentially a good thing but the implementation of it currently leaves a lot to be desired.

This is _not_ a consumer product.

<comment><USER3> Conversely one could argue that having access to your SNP data, accompanied by some good learning material and software (and 23andme do a pretty good job here) is the best way for people to get a better understanding of this kind of data.

You have to start somewhere and the underlying technology is now mature enough that having a commercial company (or companies in this case) take a lead in writing material and software targeted at the consumer is a good thing.

<comment><USER2> Maybe. All I see is that people that are worried about some heritable disease are out $400.

I spent the better part of a year to brush up on genetics (because my former gf worked in a university lab doing a project for her PhD) and this stuff is not something you explain in a leaflet or booklet.

That may be _just_ enough to get them to interpret the data.

Also note that their 'genetics 101' simplifies things to the point that it should qualify as entertainment.

To really interpret this data and to get some idea of what the (potential) impact on your life is you need a lot of background and possibly a multi-hour sit-down with an expert in the field.

And that's all still before looking at things like should a commercial entity have access to your genetic information ?

Note that their privacy policy explicitly permits them to make available to third parties your genetic data, but 'without your name or any other account information'.

Those people should know better, after all what is more identifying than your genome...

And what if they get bought out by some third party that decides to play fast and loose with this information.

<comment><USER3> I am not denying the complexity or the potential risks, but I also believe that this is the kind of information people should understand, and can take to a genetic counselor and discuss. The information is well curated, updated, you get your raw data.

I would insist on that policy. Genetic information in it's individuality is of no use. With a large enough pool with associated traits, it's ideal for research and that's why they have it.

I live in this world and yes, the data are complex, but there are some very smart people working on making this more accessible (including at 23andme). If we want people to be curious about genetics, make an effort to understand the information, and also understand the implications, if a commercial entity comes along to give this a shot in the arm, it's a good thing. If it's too early, the market will make that decision.

</comment></comment></comment></comment><comment><USER4> Two week old article. Why is it getting published now?

Just to add some content: 23andme never did complete genomes, they only targeted specific markers. As a consumer product there really wasn't anything compelling, and it was too expensive.

They've now changed their business strategy from consumer to B2B and let go most of the consumer folks.

<comment><USER2> > Why is it getting published now?

Because posting techcrunch articles is guaranteed karma ;)

> They've now changed their business strategy from consumer to B2B and let go > most of the consumer folks.

So much for the 'me' part then.

Even if they didn't do 'whole genome' there might be a freezer there with the samples that they took in, so in the long run when whole genome analysis becomes cheap enough they might do a 'replay'.

They reserve the right to share your genetic data but at the same time they say they will not share your 'personal information' without your consent.

That's the weirdest thing I've ever seen in a privacy statement, for one it is contradictory (genetic data being rather personal), and on top of that it gets the priorities completely wrong.

The only good privacy policy for a company like this would be 'We _guarantee_ that we destroy all information you have supplied us with after we send you your results, this includes any samples and personal data'.

Anything less simply won't do.

</comment></comment><comment><USER5> I think this is a company that is only being propped up by the capital injected to be honest.

<comment><USER2> It is essentially the hobby project of the spouse of a very wealthy individual, but that doesn't mean it couldn't take off.

The lay-offs are a sign that not all is well though, when their #2 took off a couple of months ago that was a clear indication that things were not going in the right direction.

What puzzled me at the time was why they were doing their best to spin it rather than to just be up-front about the reasons. If 23andme was a an all-out success you can bet that their first-in staff would not be leaving.

</comment></comment><comment><USER6> I've always wondered what good it would do me to know that I have a certain percentage chance of getting Alzheimer or Parkinson disease. In some situations I could understand the use (certain highly predictable cancers), but in most cases I don't see the point.

I already live a healthy lifestyle. I eat right. I don't smoke. I exercise 2-3 times a week. I don't think I could try much harder.

To me it's kind of like trying to figure out when I'm going to die. Why do I want to know this? Just let me live my life.

These are some reasons why I don't think people buy these kinds of tests. That and the fact that it costs $500!

<comment><USER7> A former colleague of mine was working on an EU project to gauge what outcomes this technology would have. The Medical Associations were and are objecting to it being rushed into the public domain without regulation. The Doctors themselves don't have the knowledge to interpret what those results might mean for you in the long-term and the scope for over-diagnosis is huge. And a market for quacks in this field is also sadly inevitable.

From what I hear this is also turning out to be the opinion from US health professionals. Though I guess the health insurance companies would love this technology...

</comment></comment><comment><USER8> Two weeks old. Not News.

<comment><AUTHOR> It was to me.</comment></comment>


Differences Between Perl 5 and Perl 6 - vezzy-fnord http://design.perl6.org/Differences.html <comment><AUTHOR> To reuse the recent slogan, the only thing that changed is everything.

It's a new language with its own internal logic.

I used (and still use) Perl 5 because it made some specific things easy to make (various processing of data). I still don't know for which kind of tasks Perl 6 is the best choice. But maybe the language is simply too young for it to be obvious?

<comment><USER2> Perl6 is really a strict supserset of Perl5 in this way.

Perl6 is _at least_ as good as Perl5 for everything Perl5 is good at.

But Perl6 is way, way more than that.

<comment><USER3> _Perl6 is at least as good as Perl5 for everything Perl5 is good at._

No. It is not. Don't believe me? Here is a short list of classic selling points for Perl 5 that Perl 6 clearly loses on.

1\. Performance. Back in the 1990s, Perl hit a sweet spot of being quick for development while still being fast enough for a wide variety of tasks. It still performs reasonably well versus other scripting language. Every time I looked at Perl 6 it failed to be fast enough for anything interesting to me.

2\. Library support. Any argument for Perl quickly lists CPAN. It is a large part of why people adopted it in the first place. Perl 6 loses to every major language, including Perl 5, on this. If you need to do anything from connect to a database to sending email, Perl 6 loses.

3\. Backwards compatibility. Perl has a history of being careful with backwards compatibility, starting with Perl 1's unit test suite back in the mid-80s. (It would be decades until other language communities caught up.) Most organizations currently using Perl 5 use it because they already have existing code in Perl 5. Their primary motivation for staying on Perl is that they don't want to rewrite their code. Perl 6 requires a rewrite of their code.

<comment><USER4> 1\. Can't argue with this.

2\. With Inline::Perl5[1] this should be a non-issue. The short code snippet in the README at the github repo referenced should give you the idea.

3\. Having not been released yet, Perl 6 has zero history on backwards compatibility, so there's no way to assess whether it's relatively better or worse than Perl 5.

That said, I don't necessarily support the original statement that Perl 6 is a superset and at least as good as Perl 5 at everything. I just wanted to address your specific statement regarding library support.

1: [https://github.com/niner/Inline-Perl5](https://github.com/niner/Inline- Perl5)

<comment><USER5> > Having not been released yet, Perl 6 has zero history on backwards > compatibility, so there's no way to assess whether it's relatively better or > worse than Perl 5.

Well, uh, it's not backwards-compatible with Perl 5. Yes, yes, I know, "It's a different language." So why is it called "Perl 6?" The name is very much designed to tell us "this is the one after Perl 5." Insisting that everyone evaluate it as a greenfield language is an incredible insistence upon cognitive dissonance.

<comment><USER6> Meh. Read it as "This is the 6th perl Larry's designed".

It is the one after 5, in that sense.

The naming is not perfect but it is, at this point, a little late to do anything about that.

<comment><USER5> But the other five aren't different languages, they're different versions of the same language. (And if it weren't for the Perl 6 debacle I think what we currently call Perl 5.22 would be something like Perl 6 or even Perl 8 or 9.)

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> Do the changes to the regular expression syntax seem woefully under-publicized here?

        Was:    $str =~ m/^\d{2,5}\s/i
        Now:    $str ~~ m:P5:i/^\d{2,5}\s/ # :P5 indicates Perl5-compatibile regex syntax

        Was:    $str =~ s/(a)/$1/e;
        Now:    $str ~~ s:P5/(a)/{$0}/;    # $1 is now $0

I actually find the change from $1 to $0 particularly disturbing, since that is well standardized and group 0 normally refers to the whole matched pattern (though Perl had to give it a different name and for some reason has always discouraged its use).

They also provide no information here on the non-"Perl5-compatible" regexes, but they do talk about it a bit in Specification 5: [http://design.perl6.org/S05.html](http://design.perl6.org/S05.html) There are some really interesting changes there, and we'll see what comes of this -- PCRE has become a de-facto standard, so I'm curious to see what happens as Perl moves away from "PCRE-compatible" syntax.

<comment><AUTHOR> It seems "disturbing" is the good word here, as the kind of changes in the semantics or notation between Perl 5 and Perl 6 almost guarantees that most potential users would have to decide if they are going to reject using one or another: The need to "read differently" on the distant look similar code demands too much from the people who'd just like to "get the job done."

What's that $1? Well it's the first match if it's Perl 5 but it's a second match if it's Perl 6. But don't worry, the other things in expressions also have sometimes other meanings.

<comment><USER8> don't use $1, use named captures or list assignment. qr/\$\d+/ is a oneliner thing or a code smell.

<comment><USER9> It's not a smell if you are doing an asston of text munging. Rather than write one ginormo regex I prefer a sequence of short actions. It's easier to debug and reason with.

<comment><USER10> Use named captures or list assignment. $1 $2 etc are already a code smell in Perl 5. Action-at-a-distance is the enemy.

<comment><USER9> That's needlessly verbose if I am descoping $1 within a line or two. I agree one must be careful.

</comment></comment></comment></comment></comment></comment><comment><USER11>
      Was:    my $len = length($string);
      Now:    my $len = $string.chars;

My intuitive interpretation of that syntax would be "given a string, return a sequence containing its characters, in order".

Or is that overlading on return type, and do you get the behavior I would find intuitive if one assigns to an array?

Even if that is true, I find that ugly. I would prefer more explicit code, for example

       my $len = $string.chars.#;

Similarly:

      Was:    $#array+1 or scalar(@array)
      Now:    @array.elems

(All IMHO)

<comment><USER12> this is a typo error actually: Now: my $len = $string.length;

[http://doc.perl6.org/type/Str#method_length](http://doc.perl6.org/type/Str#method_length)

EDIT: I don't see any contact section on their page to let them know

<comment><USER13> length is gone. See:

[http://design.perl6.org/S29.html#Obsolete_Functions](http://design.perl6.org/S29.html#Obsolete_Functions)

"length()

This word is banned in Perl 6. You must specify units. In practice, this probably means you want Str.chars(), although it could be Str.bytes(), or even something else. See S32-setting-library/String for details."

<comment><USER14> That's an interesting design decision, and I suppose it solves the Python problem of "I want to operate on any iterable, except for strings which are technically iterables but I want to treat them as atomic elements."

(Incidentally, does Python 3 have a solution to this problem?)

</comment></comment></comment></comment><comment><USER15> Note that its more or less "how to write perl 5 in perl 6"

Sorta like the old F2C, Fortran to C, converter. You don't get idiomatic C, but you do get a fortran program that compiles and runs as a C program, more or less.

Some parts of perl6 are annoyingly urbit-y, come on, you guys just had to rename ARGV and rename scalar and the for-foreach transition is ridiculous. I suspect at least a decade of people pointing at items and verbally calling them scalars and so on. Urbit is cool but is most famous for feeling the need to rename everything.

Here's a slide stack for going from MOOSE (more or less the most popular "real" perl5 OO implementation, technically an "add on" for perl5 although many people will only program perl5 with MOOSE) to perl6 OO. At least toward the middle there are adjacent slides of "this is how you do it in MOOSE" "this is how you do it in perl6"

[http://www.slideshare.net/lichtkind/p6-oo-vs-moose- moo](http://www.slideshare.net/lichtkind/p6-oo-vs-moose-moo)

I'd like to read a book titled something like "Modern Perl; For Perl 6.0"

The transition will be such fun; I have some business logic libraries where I'll have to replicate behavior and test suites simultaneously for awhile in MOOSE/5.0 and in 6.0.

(response to unfortunately hellbanned berntb question about replication, the codebase is huge and interrelated and can't be forklift upgraded all at once; its already non-trivial to do stuff in both perl and scala, and now it would be perl5 perl6 and scala all more or less interoperating)

<comment><USER16> It might be too much stress or too little coffee, but I don't understand the last part. Why would you need to replicate behaviour and tests?

You mean it will take time before you trust the "use Perl 5 code in Perl 6" bridge that is worked on, so it isn't enough to run the Perl 5 tests and then use that code in Perl 6?

(Or you think there will be bad interactions with Moose and Perl 6 OO?)

Edit: Huh, I was shortly un-hellbanned and am Hellbanned again since the previous comment didn't live up to some interesting-level standard? :-)

<comment><USER17> > Edit: Huh, I was shortly un-hellbanned and am Hellbanned again since the > previous comment didn't live up to some interesting-level standard? :-)

Maybe your comment got rescued by someone using the new vouch feature?

</comment></comment></comment><comment><USER18> In multiple organizations I have primarily seen Perl applied to very large, complex and established code bases that also make significant use of things like reading/writing Perl data structures.

Simply put, there will _not_ be any grand push to re-sigil every stored data structure and make massive, sweeping changes to nearly every line to end up with the same program that looks, if anything, arguably less intuitive.

This shows a profound unawareness of how Perl 5 was actually used in practice.

<comment><USER19> Perl 6 developer here.

Perl 6 is not an automatic upgrade path from Perl 5. We don't expect anybody to rewrite huge amounts of Perl 5 code in Perl 6. Perl 5 is going to be maintained separately.

We do provide tools to augment Perl 5 code with Perl 6 code, and the other way around. There's Inline::Perl5 for Perl 6, and Inline::Perl6 for Perl 5.

<comment><USER20> Is there a Perl 5 EOL being discussed?

Or, if Perl 5 and Perl 6 really are two completely separate languages separated by a similar syntax, then what comes after Perl 5 in the Perl 5 language? Will it forever be 5.N, 5.N+1, ...? (And, I guess, will there forever be 6.N, 6.N+1, ...?)

<comment><USER21> Its ridiculous to imagine any EOL for Perl 5, given how large that ecosystem is. Co existence is the way forward.

I think a good deal of Perl 6 features will seep into Perl 5 over time, while maintaining backwards compatibility.

<comment><USER22> i think if Perl6 becomes popular enough, most perl5 devs and shops will just move to perl6

while perl6 is truely a new languages not an upgrade to perl5

perl5 devs for a while have been under pressure to make perl5 more popular, using a popular language adds a padding of security, a cushion, perl5 devs are surely hungry for this

and if you like perl5 you will love perl6 as they seem to share the same values both languages that are syntax heavy, and languages that work the way you do, instead of forcing you to think a certain way .. like say functional languages

Perl6 is probably the complete opposite of say .. Haskell, but I believe in a good way ... it will be popular, i think it will be an agile language .. and many devs will love the flexibility

</comment></comment></comment></comment></comment><comment><USER23> So what you're telling me is that perl no longer automatically applies _flatten_ to nested lists? Yay!

<comment><USER24> That was never a problem you just used:

$x = [ [1], [2], [3] ];

Instead of:

@x = ( (1,), (2,), (3,) );

I always found it quite convenient to be honest.

</comment></comment><comment><USER2> I really like how the various try related constructs, for instance, CATCH, are lexically scoped inside the overall try block. This is a very logical and useful simplification I think.

</comment><comment><USER19> See also [http://doc.perl6.org/language/5to6](http://doc.perl6.org/language/5to6) (which is well maintained).

</comment><comment><USER25> Everything is a reference! Nice!

<comment><USER26> Not really if you car about efficiency.

<comment><USER27> The opposite is true. In Perl 5 passing a hash to a function requires mallocing a new shallow copy of that hash. In Perl 6 this would be the equivalent of passing a reference to the hash, which would only require mallocing a scalar.

</comment></comment></comment><comment><USER28> Please see also cperl, a perl11 (=5+6), which is an extended perl5 variant with many perl6 features and syntax extensions. Only the perl6 syntax which violates perl5 cannot be supported.

[http://perl11.org/cperl/STATUS.html](http://perl11.org/cperl/STATUS.html)

</comment><comment><USER29> One thing that bugs is that perl5 code could potentially run with perl6. Is there any mitigation for that?

<comment><USER30> It was explicitly not a design goal to be backwards compatible.

"In exchange for breaking backwards compatibility, at least at the language level, Perl 6 offers plenty of high-powered language concepts that Perl 5 didn't support..."

[http://www.perl.com/pub/2006/01/12/what_is_perl_6.html](http://www.perl.com/pub/2006/01/12/what_is_perl_6.html)

<comment><USER31> That's not what I read the GP post as saying though. Rather, that the perl 6 interpreter could run code written as perl 5 (and never ported). With their different semantics, the results would be unpredictable and often undesired.

<comment><USER32> From this document, the chance that a perl 5 script is syntactically valid (but semantically different) perl 6 is awfully low. Now if the errors that arose said that you might be trying to execute perl 5 as perl 6, that'd be cool ;)

<comment><USER33> There are numerous error messages that highlight perl 5 isms

e.g.

      > say $#array;
      ===SORRY!=== Error while compiling <unknown file>
      Unsupported use of $#variable; in Perl 6 please use @variable.end

and

      package foo;
      ===SORRY!=== Error while compiling <unknown file>
      This appears to be Perl 5 code. If you intended it to be Perl 6 code, please use a Perl 6 style declaration like "unit package Foo;" or "unit module Foo;", or use the block form instead of the semicolon form.

<comment><USER17> Yes, these suggestions are among my favorite things about Rakudo.

There are a lot more, too. Another example:

        > 1 == 1 ? "sane" : "insane"
        ===SORRY!=== Error while compiling <unknown file>
        Unsupported use of ? and : for the ternary conditional operator; in Perl 6 please use ?? and !!
        at <unknown file>:1

Or:

        > loop(my $i = 0; $i < 10; $i++) { say $i }
        ===SORRY!===
        Word 'loop' interpreted as 'loop()' function call; please use whitespace around the parens
        at <unknown file>:1

Easily the nicest error reporting I've seen in a compiler.

</comment></comment></comment></comment></comment></comment><comment><USER34> >any two valid identifiers may be joined with a single ' to form a longer identifier

Oh no.

<comment><USER35> It gets worse. "Hey, let's double down on weak typing!": [http://examples.perl6.org/categories/cookbook/01strings/01-0...](http://examples.perl6.org/categories/cookbook/01strings/01-00introduction.html#Strings_as_numbers)

      print  +""     ; # a num-ified empty string evaluates as 0
      print  "1" + 1 ; # 2

The opportunity for insidious bugs (for the benefit of not having to type 'int("1") + 1' those times when you really need to do that) is a terrible tradeoff, in my opinion.

In before "but Python...!". Python is strongly typed.

      >>> "1" + 1
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
      TypeError: cannot concatenate 'str' and 'int' objects
      >>> 1 + "1"
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
      TypeError: unsupported operand type(s) for +: 'int' and 'str'

<comment><USER4> This is not weak typing. Perl (both 5 and 6) use operators to define type context. It's entirely unambiguous whether you want to add something or concatenate strings because _they are different operators_.

      "1" + "1" # 2
      "1" . "1" # 11 (~ to concatenate in Perl 6)
      1 + 1 # 2
      1 . 1 # 11

When you perform a string operation on a numeric type, it stringifies the number, when you do a numeric operation on a string, it numerifies the string (and warns if it can't be cleanly done).

This isn't a typing problem, this is a commenting on language without actually knowing how it works problem. But that's nothing new, people have been doing that to Perl for a long time now.

<comment><USER35> Ah, sorry. Despite having used Perl for years, I'd forgotten how much emphasis it puts on concepts that invoke a high cognitive load.

<comment><USER4> On the contrary, I think there is a higher cognitive load in having to reason about the behavior that results from using the + operator between two types when their types may differ. e.g., the Wat talk[1].

To be clear, there is a cognitive load to both. One is based on keeping correct type of the variables you are using with the + operator, the other is knowing that numeric addition and string concatenation are not the same thing, and have different operators. Since string concatenation and numeric addition _really aren 't the same thing_, and don't share the same properties, I prefer the different operators.

1: [https://www.destroyallsoftware.com/talks/wat](https://www.destroyallsoftware.com/talks/wat)

<comment><USER3> _On the contrary, I think there is a higher cognitive load in having to reason about the behavior that results from using the + operator between two types when their types may differ. e.g., the Wat talk[1]._

The simple fact that you run out of operators limits how many types can usefully exist. In Python you use + quite reasonably for integers, floats, strings, and arrays. By contrast raw Perl has equivalents only for numbers and strings, and is unable to distinguish between integers and floats. That is why Perl doesn't do exact arithmetic with large integers.

But, you say, Perl allows you to use bigint? Well, yes. And that is internally implemented with the magic of overload. Using that you can have any number of types which overload + to do what they want, and you have all of the same issues as arose in the Wat talk.

It took a long time for me to be convinced, but I am now convinced that typed variables and untyped operators (like Ruby and Python do) is strictly better in practice than untyped variables and typed operators (like Perl does by default). And as for the complexity, it is just something you have to live with.

Want evidence? [https://mitpress.mit.edu/sicp/full- text/sicp/book/node48.htm...](https://mitpress.mit.edu/sicp/full- text/sicp/book/node48.html) discussed this exact problem some 30 years ago. I do not believe that the intrinsic problem is being solved any better today than then.

</comment></comment></comment></comment></comment></comment><comment><USER36> For someone using Perl 5 as a replacement for bash scripting, is it worth making the jump to Perl 6?

P.S. Why not name this language something new? -- Nicklaus Wirth didn't name Modula-2 Pascal 2.

<comment><USER37> _Nicklaus Wirth didn 't name Modula-2 Pascal 2_

And that was a big marketing mistake. Do you think C++ would have been anywhere near as successful if it had been called Z?

<comment><USER36> So its a marketing decision? That's depressing.

</comment></comment></comment><comment><USER38> And Perl6 has now invented its own word, "twigil"? Sigil was already a bad enough word. I used Per5 for a ton of production code years ago, and I've met Larry Wall and he is a brilliant and friendly person, but I am sad to say there is absolutely zero chance of Perl6 being adopted for a serious production software project, probably ever. It doesn't bring new capabilities, it's just different, and in a mostly-worse way.</comment>