Thoughts on Rule-Based Programming - <AUTHOR> http://www.eblong.com/zarf/essays/rule-based-if/

<comment><USER2> The backward subject-verb semantics in the OO examples are typical anti-OOP straw men. It doesn't have to be Room.look() and Object.examine(), when it could simply be Player.look(Room), Player.examine(Object), Player.put(Meat, Basket), etc. Don't begin with a design pattern then write code around that poorly choosen pattern. Instead write your core logic expressions first then it becomes obvious what the structure of each object ought to be.

</comment><comment><USER3> Prolog isn't all that weird and a pretty good language for adventure ages, see: [http://www.csc.villanova.edu/~dmatusze/resources/prolog/slee...](http://www.csc.villanova.edu/~dmatusze/resources/prolog/sleepy.html) [http://www.csc.villanova.edu/~dmatusze/resources/prolog/spid...](http://www.csc.villanova.edu/~dmatusze/resources/prolog/spider.html)

<comment><USER4> Speaking of Prolog, I often run into situations where I would love to have some semi-dialect of prolog available at runtime (ie: use regexes for simple string problems, use prologexes for simple logic problems).

Are you aware of any prolog-like languages that integrate well with other programming languages?

<comment><USER5> I don't know of any (outside of semi-toy Prologs in _On Lisp_ and PAIP), but writing a Prolog dialect designed for embedding (in a similar manner to Lua) is near the top of my list of future projects. Prolog can be really awkward for some things (most notably IO / side effects), but for certain problems, its approach is an excellent fit.

Constraint programming (often bundled with Prolog) also deserve far more attention, IMHO.

<comment><USER6> For CP, GECODE is C++, which makes it very easy to bundle /if/ you're writing in that language.

SWI-Prolog can be used embedded - but I haven't any experience with the results.

<comment><USER5> I'm reading Guido Tack's thesis now, actually, trying to get a better sense of how constraint systems are implemented.

SWI is my usual Prolog runtime, but I haven't tried running it embedded. I really like Lua's design principle of, "The language doesn't handle this particular thing well, but since it's embedded, it _doesn't need to_ \- keep the language small". A mini-Prolog could focus on what it does best, have a simple C API, and leave the rest to the surrounding language.

</comment></comment></comment></comment></comment><comment><USER7> Thanks so much for posting this, it brings back fond memories of playing IF games as a kid (wish I still had that much free time on my hands). Saved to InstaPaper for later reading.

</comment><comment><USER8> This sounds exactly like the semi-secret language that's been floating around in my head for months!</comment>


Ethereum Messaging Using OrbitDB and IPFS - <AUTHOR> https://medium.com/originprotocol/introducing-origin-messaging-decentralized-secure-and-auditable-13c16fe0f13e <comment><AUTHOR> If you want to see it in action, you can also try it out on the Rinkeby network at [https://demo.originprotocol.com](https://demo.originprotocol.com)

All the code is open-source too if you want to see how it works.</comment>


Ask HN: Should I fire my PM, my engineer, or myself? - <AUTHOR> Purely hypothetical (of course): Small early startup. PM is driving the engineer crazy by being hyper-agile -- changing design (not architecture) as he gets an idea into his head. (Using appropriate issue management and kanban, but on a daily -- or faster! -- not bi-weekly basis.) Meanwhile, engineer is griping and complaining about this -- wants a project plan, or at least a few days worth of one. OT1H, one could say to the engineer, this is how small early startups are. OT2H, one could tell the PM to work on a more deliberate plan/pace. OT3H, I could become a woodworker. What do you recommend? <comment><USER2> Mate your PM isn't managing very well if he can't get his thoughts on the same path for more than a few days. Tell your PM to sit down...work his shit out...if he thinks he needs to change his mind tomorrow...tell him he better have bloody good reason a. to change his mind. and b. why he didnt think of that reason the day before when he put the plan in place. Your PM is screwing you.

</comment><comment><USER3> Changing the design on a daily basis doesn't seem like a winning strategy - the engineer will never be able to complete anything since they're chasing a moving target, and so you'll never get an MVP that you can actually give to customers and get feedback on.

I don't think you necessarily need to fire your PM, but sit down with him and ask him to slow down. New ideas are great, but if he thinks about them for a while before suggesting them to the engineer, it will weed out some of the bad ideas before they get implemented. (Maybe the PM should switch to decaf coffee or get more sleep?)

If this continues, your engineer will burn out and/or quit.

<comment><USER4> This. I'm not usually a fan of scrum, but this is one of the problems it can help solve. You can decide to zig or zag every couple weeks at the end of a sprint, but for those couple weeks you'll hold steady in one direction so that you can make reliable progress (and force the people making changes to sleep on them and bundle them up.)

Frankly, though, your PM sounds ineffective - especially in a kanban system. Part of their job is making sure there's a shippable MVP SOON that you can THEN iterate on IF improving it at that point will have more business impact than building the next feature will. And they're clearly undermining that effort.

</comment></comment><comment><USER5> Is it just the 3 of you? Why do you have one PM for one engineer? What is your role? What value do frequent changes bring? Is there profitability in sight when sticking with a plan?

Ask yourself hard questions. If you're coming on an online forum to offload the responsibility of having a hard conversation with a team member due to a fear of hurting someone's feelings, maybe you should fire yourself.

<comment><AUTHOR> I'm not avoiding the conversation. I'm trying to decide which is right. Seems like everyone agrees that the PM is wrong. Now I can have the hard conversation.

</comment></comment><comment><USER6> What is your role in the startup?

<comment><AUTHOR> Hypothetical startup (of course): I conceived the product, hacked the (fully working!) prototype co-founded the company, hired the engineer to harden the "easy" part of the product. I do the AI.

<comment><USER6> I mean organizationally - do you see yourself as the CEO? The other two people have clearly defined hats (PM / engineer), but it is unclear what hat you are wearing.

<comment><AUTHOR> I would like to devolve to being the AI engineer in an engineering team that I (initially) build around me. Eventually someone else - a CTO - would build it up further. Thus the PM, who is supposed to be working with the engineering team (at the moment just a MERN engineer and me) and first users to turn my (fully working, but ugly) prototype into a scalable, usable product.</comment></comment></comment></comment>


On One Laptop per Child - <AUTHOR> http://pixelqi.com/blog1/2009/07/28/on-one-laptop-per-child/ <comment><USER2> Am I the only one who thinks that for some children, the last thing they need if for somebody to shove a computer on them?

I use computer every single day in my professional and personal life. I love them. But I am _eternally_ grateful to my parents that they didn't give me unlimited access to one till High School.

I would have missed out on untold hours of playing outdoors, building things, taking things apart and generally just learning about the world if somebody had shoved a laptop at me when I was 8 or 9.

<comment><USER3> That's goes along with a point people often miss. Giving a computer to a child isn't going to automatically impart vast amounts of wisdom. A computer much like a book or a pencil is a tool of learning. The child still has to use it in the right way to learn anything.

It's always irked me a bit when people try to pin low achievement in schools on lack of computers.

</comment></comment><comment><USER4> On the topic of flat out silly defenses I think this takes the cake. A few things...

First, supposedly Mr. Negroponte "ended the discussion of whether computers belong in children's hands". Umm...Does anyone remember this discussion? Because I can't recall a time where people were saying kids shouldn't have computers. Who exactly was making that argument?

Second, he supposedly "answered the question of whether inexpensive computers can be useful." But the very argument that's coming up more and more lately is one that points out these computers weren't terribly useful because they had tons of flaws. Don't get me wrong, I believe inexpensive computers can be useful but I don't believe that was proven by Mr. Negroponte.

Finally, note that rather than give actual accomplishments she quotes a projected figure to justify the actions of OLPC. In my life I've learned that people who quote projections as accomplishments are usually the ones who have no actual accomplishments to point to.

In the end, this post is nothing more than misplaced hero worship.

<comment><USER5> Mary Lou Jepsen has a list of accomplishments pages long, and is one of the most brilliant people I've ever met. You should be careful about tossing around vague inanities like "… usually the ones who have no actual accomplishments …" which aside from sounding pretentious don't add in the least to the discussion.

In any case, I think you misunderstood: this is the projected number of cheap netbook-like laptops which will be sold (that's between several companies), not the number of XOs, and the numbers are reasonably reliable a year or two out, because they actually have to gear up to manufacturing these things at scale well in advance of making them.

I think it's without question that OLPC and Nicholas's pushing on the public consciousness, and as importantly on hardware vendors, has sped by at least a few years the pace of cheap laptop production. There really weren't any sub-$400–500 laptops before Nicholas was talking about them 4–5 years ago, and now there's an exploding market. Most hardware manufacturers (not to mention pundits and "technologists" and so forth) laughed in his face because they said the margins would be too low to make sense, and the components couldn't be had for the price. Then at some point he convinced Quanta to jump on board, and everyone started looking really closely.

(disclaimer: I spent a year at OLPC as an intern)

<comment><USER4> I note you don't list said accomplishments. Good intentions are not in themselves accomplishments. Sure the media will love you for "creating a $100" computer but if that computer doesn't actually help who you said it would that isn't an accomplishment..

Bottom Line: Defending one's accomplshments by saying they have accomplishments is circular logic

If anything I'm more weary of her for using a non-profit to develop technology she's now trying to sell while all the while selling flawed, next to useless computers to poor kids. (I suspect her intentions were good for the record all I'm saying is the facts paint a picture that is suspect)

<comment><USER5> Are you for real?

<http://en.wikipedia.org/wiki/Mary_Lou_Jepsen>

Quoting from there, since it's easier: "She co-created the first holographic video system in the world at the MIT Media Lab in 1989 [...] At OLPC, notably, Jepsen invented the laptop's sunlight-readable display technology." You don't get put in charge of the technology for displays at Intel (her job before OLPC) by being unaccomplished.

Here's a (non-exhaustive, I think) list of her patents: [http://www.google.com/patents?q=mary+lou+jepsen&btnG=Sea...](http://www.google.com/patents?q=mary+lou+jepsen&btnG=Search+Patents)

Several of those, like the OLPC dual-mode display, are _really_ clever (better than anything else of the kind, feasible to manufacture on existing fabs, no more expensive than existing displays, etc.).

Also, your implication that her involvement with OLPC was "suspiciously" predatory (or for that matter the implication that OLPC on the whole is some kind of exploitative scam) is a cheap insult below the expected level of this forum. Really bright high-level engineer–executives who take out a few years of their careers to work at well below pay grade for non-profit companies trying to change the world should be lauded, in my book.

<comment><USER4> This is exactly my point. Your argument boils down to the OLPC is great no matter what so anyone who dares question it or her is wrong by default.

For the record, this didn't start out as an attack on Ms. Jepsen who I said probably had good intentions. My point is people like her go around thinking their good intentions mean something without end results. So they end up pushing things like the OLPC which hurt more than they help. Then they laud themselves and their partners for having been such great people and done such a great thing and shade their eyes from all the flaws.

Every reports I've heard has said the OLPCs have major hardware and software problems and my own died pretty shortly after I opened it up. Selling PCs like that to third world countries deprives those kids of actual working PCs which in my book is a bad thing

Also, I didn't mean to malign Ms. Jepsen's technical accomplishments which are impressive. But when I referred to "accomplishments" I was talking in the sense of "making products that make a difference in the word" which is exactly what I think she thinks she accomplished and which I dispute.

(I can't reply to your comment below for some reason but let me just say this. The arguments that have been made in the last few days that suggest the OLPC is flawed are perfectly valid. The only people I've seen deny them are people like you who worked for the project. So assuming those flaws are real (again, I'm living proof since mine broke 2 weeks out) imagine a country who bought 1000 OLPCs for 1000 kids rather than 500 Windows or Linux PCs. Now if those OLPCs are flawed than those kids have been deprived of computers. The fact that you can't see that just goes to show your hero worship of a project that has clearly not met it's goals)

<comment><USER5> > _"This is exactly my point"_

Actually, no, you shifted the goal posts twice.

> _"Your argument boils down to the OLPC is great no matter what."_

You're putting words in my mouth. Where did I argue that?

> _"hurt more than they help"_

Have any evidence (or even reasoning) to back that up?

> _"shade their eyes from all the flaws"_

Do you have any evidence that Mary Lou has done that? Nicholas has, in public perhaps (though I'd be cautious to say that even).

> _"deprives those kids of actual working PCs"_

Can you back that up with a plausible explanation? As far as I can tell, not a single person has been "deprived" of a PC because of the OLPC project, and you're the only one who is suggesting so.

* * *

This is a very narrow discussion, in which you insulted and maligned someone with no cause, and I called you out on it, and then you tried to make it stick rather than apologizing as would be appropriate for a reputable discussion. I haven't made any claims about everything else being great, or anything of the kind, and I don't even really see how you're reading that into any of my statements.

* * *

Edit:

> _" people like her go around thinking their good intentions mean something > without end results."_

I also want to push back on this point. First, good intentions + deliberate action really does mean something, even in the unlikely even that they have no result. I personally think there is nothing more important in the world than the combination of good intentions plus the willingness to back those up with action. (An only slightly joking quesion: What have you done for the children of the world recently?) But second, the whole point of Mary Lou's post is that there _has_ been a result. The "netbook" market is growing by leaps and bounds, and more and more children all over the place are getting their hands on various kinds of laptops, whether made by OLPC or otherwise. This was always Nicholas's primary publicly stated goal (whether he had different private goals I have no idea), and the success on that front more than justifies the effort.

</comment></comment></comment></comment></comment></comment><comment><USER6> This is Mary Lou Jepsen, the former CTO of OLPC, apparently weighing in on Ivan Krstic's criticism of Nicholas Negroponte's criticism of the OLPC's original choice of software stack.

<http://news.ycombinator.com/item?id=719048>

<comment><USER7> And she hardly adds any argument or anything, it's just like "don't touch our brave hero Negroponte, because some of his achievements, completely irrelevant to the software stack and its critisism, which is the discussed problem".

(Plus I'm really sick of disgusting pathos of next to the last sentence.)

</comment></comment><comment><USER8> Tremendous effort by Negroponte, to be sure.

I am fortunate enough to be able to afford to give my children laptops. For some reason, it doesn't seem important to do so. I guess I just don't get it.

Perhaps teaching children to read and giving them access to books is more worthy of effort than giving them laptops. Not as newsworthy or noteworthy though.

Anyway, enough of my quibbling. If more of us had the drive and will to make the world a better place that Negroponte has, surely the world would be a better place.

</comment><comment><USER9> Is this the same Jepsen who "took the money and ran"? "The money" in this case being the LCD technology she developed - _funded by our donations._

<comment><USER6> The patents have stayed with OLPC.

 _"Pixel Qi -- my new company -- is now licensing my inventions from OLPC. This isn't an OLPC employee benefit, it's a deal I created with OLPC and Pixel Qi, and the benefit will go to OLPC and to the children of the world, lowering the price of the laptops, and thus allowing more kids to get laptops."_

<http://www.groklaw.net/article.php?story=20080107182525297>

<comment><USER9> > The patents have stayed with OLPC.

This tells us nothing about how they will be licensed. They will be available for free use by the well-connected, but certainly not to you or me.

<comment><USER10> Since you and I don't build LCD panels in our spare time, I suppose that may be just as well.</comment></comment></comment></comment>


Can we please make HN mobile friendly? - <AUTHOR>

Reading HN on my phone is a pain in the butt. It would be really great if HN was mobile friendly so I didn't have to constantly pinch zoom in to read and click on posts. <comment> jack-r-abbit _A responsive version that works better on mobile is coming_ [1]

[1][https://github.com/HackerNews/HN/issues/104](https://github.com/HackerNews/HN/issues/104)

</comment><comment><USER2> There are quite a few decent mobile apps for reading HN. I use HN reader on Android, which has a few problems, but I actually like better than the website in some ways. For example, I can collapse comment trees.

</comment><comment><USER3> Seriously. HN is too good to stay desktop-only.</comment>


Show HN: See your GitHub followers based on rank - <AUTHOR> http://www.github-followers.com

<comment><AUTHOR> One weird thing that I've discovered is that there seem to be no correlation between who I consider a great hacker and how many followers they have in top 10k.

For instance maxogden [1] and mafintosh [2] are both in top 1k but have only 6 top 10k followers each. Where as I [3] are not even in top 10k and have 9 followers in top 10k.

[1] [http://www.github-followers.com/maxogden](http://www.github- followers.com/maxogden) [2] [http://www.github- followers.com/mafintosh](http://www.github-followers.com/mafintosh) [3] [http://www.github-followers.com/watson](http://www.github- followers.com/watson)

</comment><comment><AUTHOR> Ok, so apparently I've exceeded the GitHub API rate limit - Trying to fix it ;)

<comment><AUTHOR> I've got 4999 requests remaining now, so I should be fine for a while again. Go easy on her ;)

If anyone knows if there's a way to increase this number please let me know :)</comment></comment>


IBM Watson: The new Internet search engine? - <AUTHOR>

What would the Internet be like if IBM Watson was doing the searching for you? Time to start thinking about shorting that Google stock? <comment><USER2> Watson needs to at least start off with a large curated dataset. Without that it would be very vulnerable to the inevitable influx of spam.

Spam and content farms are a big problem for Google, because of the market share it's in a spot where it can't curate content.

Watson tech would be best applied to specific domains, like Wolfram Alpha, which hasn't exploded in popularity. People prefer one good enough search destination for simplicity, Google's calculator is good enough for those searches.</comment>


CrunchPad Specs - <AUTHOR> http://www.dchieng.com/crunchpad-specs-revealed/ <comment><USER2> First $200 (I didn't believe it) - then $300 (I figured he wouldn't risk raising the price twice; - now its $400, unless he hits a grand slam and gets this thing into a carrier or even multiple carriers; and subsidized for $199 with 2/yr mobile broadband contract..

<comment><USER3> Many major hardware players have some form of tablet device in the offing. This market space is going to be a competitive one.

<comment><USER4> Like who? :/ Apple??? yeah funny.

</comment></comment></comment><comment><USER5> One USB port? Dude...

You're only ever one lap-month away from wiggling any given USB port into uselessness if anything is plugged into it. Relying on a single one for the whole thing is a recipe for disaster.

Case in point, by my old Latitude C400. With no optical drive and a single dying USB port, it's one failed hard drive away from being toast. Right now I can get files onto it over the network, but I don't look forward to reinstalling those network drivers without any input device other than the keyboard.

</comment><comment><USER6> Screen resolution? RAM? Flash memory?

</comment><comment><USER7> The styling isn't actually as slick looking as I thought it might be.

</comment><comment><USER8> seems like quite a lot of feature creep. mobile internet now huh?

</comment><comment><USER9> If I ever buy a tablet it will be an Apple iPad, no matter how expensive it is.

</comment><comment><USER10> If it's using the Linux kernel, it's not mentioned anywhere. This is taking the credit from all the linux code contributors everywhere. Shame. Shame.

<comment><USER11> The GPL does not have an advertising clause.

<comment><USER9> If you take and give back you're a friend, if you don't you're a thief.</comment></comment></comment>


Lord of the io_uring: io_uring tutorial, examples and reference - <AUTHOR> https://unixism.net/loti/ <comment><USER3> One thing this writeup made me realize is, if I have a _misbehaving_ I/O system (NFS or remote block device over a flaky network, dying SSD, etc.), in the pre-io_uring world I'd probably see that via /proc/$pid/stack pretty clearly - I'd see a stack with the read syscall, then the particular I/O subsystem, then the physical implementation of that subsystem. Or if I looked at /proc/$pid/syscall I'd see a read call on a certain fd, and I could look in /proc/$pid/fd/ and see which fd it was and where it lived.

However, in the post-io_uring world, I think I won't see that, right? If I understand right, I'll at most see a call to io_uring_enter, and maybe not even that.

How do I tell what a stuck io_uring-using program is stuck on? Is there a way I can see all the pending I/Os and what's going on with them?

How is this implemented internally - does it expand into one kernel thread per I/O, or something? (I guess, if you had a silly filesystem which spent 5 seconds in TASK_UNINTERRUPTIBLE on each read, and you used io_uring to submit 100 reads from it, what actually happens?)

<comment><USER4> I think that's a very reasonable concern. It however isn't really about io_uring - it applies to all "async" solutions. Even today if you are running async IO in userspace (e.g. using epoll), it's not very obvious where something went wrong, because no task is seemingly blocked. If you attach a debugger, you might most likely see something being blocked on epoll - but a callstack to the problematic application code is nowhere in sight.

Even if pause execution while inside the application code there might not be a great stack which contains all relevant data. It will only contain the information since the last task resumption (e.g. through a callback). Depending on your solution (C callbacks, C++ closures, C# or Kotlin async/await, Rust async/await) the information will be between not very helpful and somewhat understandable, but never on par with a synchronous call.

<comment><USER5> > Even today if you are running async IO in userspace (e.g. using epoll), it's > not very obvious where something went wrong, because no task is seemingly > blocked.

It doesn't apply to file IO, which is never non-blocking, and can't be made async with epoll. Epoll always considers files ready for any IO. And if the device is slow, the thread is blocked with dreaded "D" state.

<comment><USER6> The fundamental problem is that readiness based async IO and random access to not mix well. You'd need a way to poll readiness for different positions in the same file at the same time.

Completion based async (including io_uring on Linux or IO completion ports on Windows) doesn't suffer from this problem.

</comment></comment></comment></comment><comment><USER7> There are some benchmarks that show io_uring as a significant boost over aio: [https://www.phoronix.com/scan.php?page=news_item&px=Linux-5....](https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.6-IO- uring-Tests)

I see that nginx accepted a pull request to use it, mid last year: [https://github.com/hakasenyang/openssl- patch/issues/21](https://github.com/hakasenyang/openssl-patch/issues/21)

Curious if it's also been adopted by other popular IO intensive software.

<comment><USER8> I have not adopted io_uring yet because it isn't clear that it will provide useful performance improvements over linux aio in cases where the disk I/O subsystem is already highly optimized. Where io_uring seems to show a benefit relative to linux aio is more naive software design, which adds a lot of value but is a somewhat different value proposition than has been expressed.

For software that is already capable of driving storage hardware at its theoretical limit, the benefit is less immediate and offset by the requirement of having a very recent Linux kernel.

<comment><AUTHOR> For regular files, aio works async only if they are opened in unbuffered mode. I think this is a huge limitation. io_uring on the other hand, can provide a uniform interface for all file descriptors whether they are sockets or regular files. This should be a decent win, IMO.

<comment><USER8> That was kind of my point. While all of this is true, these are not material limitations for the implementation of high-performance storage engines. For example, using unbuffered file descriptors is a standard design element of databases for performance reasons that remain true.

Being able to drive networking over io_uring would be a big advantage but my understanding from people using it is that part is still a bit broken.

<comment><USER9> The ScyllaDB developers wrote up their take here: [https://www.scylladb.com/2020/05/05/how-io_uring-and-ebpf- wi...](https://www.scylladb.com/2020/05/05/how-io_uring-and-ebpf-will- revolutionize-programming-in-linux/)

<comment><USER10> Those benchmark results are pretty impressive. In particular, io_uring gets the best performance both when the data is in the page cache and when bypassing the cache.

</comment></comment></comment></comment></comment></comment><comment><USER11> A comment from the cat example:

>/* For each block of the file we need to read, we allocate an iovec struct which is indexed into the iovecs array. This array is passed in as part of the submission. If you don't understand this, then you need to look up how the readv() and writev() system calls work. */

I have to say, I don't really understand why the author chose to individually allocate (up to millions of) single kilobyte buffers for each file. Perhaps there is a reason for it, but I think they should elaborate the choice. Anyway, I guess the first example is too simplified, which is why what follows after is not built on top of it in any way, hence they feel disjointed.

The bigger problem here is that I don't know the author, or how talented they are. Choices like that, or writing non-async-signal-safe signal handlers don't help in estimating it, either. Is the rest of the advice sound?

<comment><AUTHOR> The author here: All examples in the guide are aimed at throwing light at the io_uring and liburing interfaces. They are not very useful or very real- worldish examples. The idea with this example in particular is to show the difference how readv/writev work synchronously vs how they would be "called" io_uring. May be I should call out the fact that these programs are more tuned towards explaining the io_uring interface a lot more in the text. Thanks for the feedback.

</comment></comment><comment><USER12> So awesome... The ring buffer is like a generic asynchronous system call submission mechanism. The set of supported operations is already a subset of available Linux system calls:

[https://github.com/torvalds/linux/blob/master/include/uapi/l...](https://github.com/torvalds/linux/blob/master/include/uapi/linux/io_uring.h)

It almost gained support for ioctl:

[https://lwn.net/Articles/810414/](https://lwn.net/Articles/810414/)

Wouldn't it be cool if it gained support for other types of system calls? Something this awesome shouldn't be restricted to I/O...

<comment><USER13> The author seems to be planning to expand it to be usable as a generic way of doing asynchronous syscalls

</comment></comment><comment><USER14> Anyone familiar with the Infiniband's approach to exposing IO via rx/tx queues [0] comment whether it seems similar to io_uring's ring-buffers [1]? How do these contrast against each other?

[0] [https://www.cisco.com/c/en/us/td/docs/server_nw_virtual/2-10...](https://www.cisco.com/c/en/us/td/docs/server_nw_virtual/2-10-0_release/element_manager/user_guide/appA.html#wp1007799)

[1] [https://news.ycombinator.com/item?id=19846261](https://news.ycombinator.com/item?id=19846261)

<comment><USER15> Very limited experience with Infiniband but it seems similar, a bit more flexible (esp recently with more syscalls supported).

Also similar to but more general than RIO Sockets of Win8+:

[https://docs.microsoft.com/en-us/previous- versions/windows/i...](https://docs.microsoft.com/en-us/previous- versions/windows/it-pro/windows-server-2012-r2-and-2012/hh997032\(v=ws.11\))

</comment></comment><comment><USER16> The site pushes really hard that you shouldn't use the low-level system calls in your code and that you should (always?) be using a library (liburing).

What exactly is liburing bringing to the table that I shouldn't be using the uring syscalls directly?

<comment><USER12> You absolutely can use system calls in your code. The kernel has an awesome header that makes this easy and allows you to eliminate all dependencies:

[https://github.com/torvalds/linux/blob/master/tools/include/...](https://github.com/torvalds/linux/blob/master/tools/include/nolibc/nolibc.h)

This system call avoidance dogma exists because libraries generally have more convenient interfaces and are therefore easier to use. They're not strictly necessary though.

It should be noted that using certain system calls may cause problems with the libraries you're using. For example, glibc needs to maintain complete control over the threading model in order to implement thread-local storage. By issuing a clone system call directly, the glibc threading model is broken and even something simple like errno is likely to break.

In my opinion, libraries shouldn't contain thread-local or global variables in the first place. Unfortunately, the C language is old and these problems will never be fixed. It's possible to create better libraries in freestanding C or even freestanding Rust but replacing what already exists is a lifetime of work.

> What exactly is liburing bringing to the table that I shouldn't be using the > uring syscalls directly?

It's easier to use compared to the kernel interface. For example, it handles submission queue polling automatically without any extra code.

</comment></comment><comment><USER17> io_uring still has its wrinkles.

We are scrambling right now to fix a problem due to change in behavior exposed to user-space from the io_uring kernel module in later kernels.

Turns out that in earlier kernels (Ubuntu 19.04 5.3.0-51-generic #44-Ubuntu SMP) io_uring will not return short reads/writes (that's where you ask for e.g. 8k, but there's only 4k in the buffer cache, so the call doesn't signal as complete and blocks until all 8k has been transferred). In later kernels (not sure when the behavior changed, but the one shipped with Fedora 32 has the new behavior) io_uring returns partial (short) reads to user space. e.g. You ask for 8k but there's only 4k in the buffer cache, so the call signals complete with a return of only 4k read, not the 8k you asked for.

Userspace code now has to cope with this where it didn't before. You could argue (and kernel developers did :-) that this was always possible, so user code needs to be aware of this. But it didn't used to do that :-). Change for user space is _bad_ , mkay :-).

<comment><USER17> It was really interesting how this was found.

A user started describing file corruption when copying to/from Windows with the io_uring VFS module loaded.

Tests using the Linux kernel cifsfs client and the Samba libsmbclient libraries/smbclient user-space transfer utility couldn't reproduce the problem, neither could running Windows against Samba on Ubuntu 19.04.

What turned out to be happening was a combination of things. Firstly, the kernel changed so an SMB2_READ request against Samba with io_uring loaded was _sometimes_ hitting a short read, where some of file data was already in the buffer cache, so io_uring now returned a short read to smbd.

We returned this to the client, as in the SMB2 protocol it isn't an error to return a short read, the client is supposed to check read returns and then re- issue another read request for any missing bytes. The Linux kernel cifsfs client and Samba libsmbclient/smbclient did this correctly.

But it turned out that Windows10 clients and MacOSX Catalina (maybe earlier versions of clients too, I don't have access to those) clients have a _horrible_ bug, where they're not checking read returns when doing pipeline reads.

When trying to read a 10GB file for example, they'll issue a series of 1MB reads at 1MB boundaries, up to their SMB2 credit limit, without waiting for replies. This is an excellent way to improve network file copy performance as you fill the read pipe without waiting for reply latency - indeed both Linux cifsfs and smbclient do exactly the same.

But if one of those reads returns a short value, Windows10 and MacOSX Catalina _DON 'T GO BACK AND RE-READ THE MISSING BYTES FROM THE SHORT READ REPLY_ !!!! This is catastrophic, and will corrupt any file read from the server (the local client buffer cache fills the file contents I'm assuming with zeros - <USER2> haven't checked, but the files are corrupt as checked by SHA256 hashing anyway).

That's how we discovered the behavior and ended up leading back to the io_uring behavior change. And that's why I hate it when kernel interfaces expose changes to user-space :-).

<comment><USER18> > in the SMB2 protocol it isn't an error to return a short read, the client is > supposed to check read returns and then re-issue another read request for > any missing bytes

This is interesting and somewhat surprising, since Windows IO is internally asynchronous and completion based, and AFAIK file system drivers are not allowed to return a short read except for EOF.

And actually, even on Linux file systems are not supposed to return short reads, right? Even on signal? Since user apps don't expect it? (And thus it's not surprising that io_uring's change broke user apps.)

So it wouldn't be surprising to learn that the Windows SMB server never returns short reads, and thus it's interesting that the protocol would allow it. Do you know what the purpose of this is?

<comment><USER17> Obviously the Windows SMB server never returns short reads, otherwise this bug would never have made it out of Redmond or Cupertino.

On Linux, pread also never returns short reads against disk files if the bytes are available, which is why no one noticed this client bug as our default io backend is a pthread-pool that does pread/pwrite calls. It only happens when someone tried our (flagged as experimental thank god) vfs_io_uring backend.

Yeah the protocol even has a field in the SMB2_READ request called MinimumBytes, for which the server should fail the read if less than these bytes are available on return. The Windows 10 clients set this to zero :-). The MacOSX Catalina client sets it to 1. So yes, the clients are supposed to be able to handle short reads.

<comment><USER18> Out of curiosity, I took a look at how the MinimumBytes (actually MinimumCount) field is used by the Windows SMB server. Interestingly, it fails with STATUS_END_OF_FILE if the actual bytes read is less than MinimumCount, which suggests to me that this is supposed to be a minimum on the (remaining) file length, not on the number of bytes that the server is able to return at the moment.

I can't find any history of MinimumCount being used in the RTM version of any Windows SMB client, so without deeper archeology the reason this field was introduced remains a mystery to me.

Regardless, I agree that the client should validate the returned byte count. But (only having thought about this briefly), I do not think a client should retry in this case--it seems to me if the client sees a short read, it can assume that the read was short because the read reached EOF (which may have changed since the file's length was queried).

<comment><USER19> Sorry to keep laboring the point :-) but the other reason I'm pretty sure this is a client bug is that the client doesn't truncate the returned file at the end of the short read, which you'd expect if it actually was treating short read as EOF.

If you copy a 100mb file and the server returns a short read somewhere in the middle of the read stream the file size on the client is still reported as 100mb, which means file corruption as the data in the client copy isn't the same as what was on the server.

That's how this ended up getting reported to us in the first place.

<comment><USER18> Yes, that's a good point. I agree that there appears to be a client bug here. From a quick glance, it appears that nothing is checking that the non-final blocks in a pipelined read are returned from the server in full.

I don't necessarily agree that retry is the right behavior though. Wouldn't that result in an extra round trip in the actual EOF case? Again, not having thought about this much, it seems a more efficient interpretation of the spec is that truncated reads indicate EOF. In that case, a truncated read as in the middle of a pipelined operation either indicate the file's EOF is moving concurrently with the operation (in which case stopping at the initial truncation would be valid) or the lease has been violated.

Regardless, I work on SMB-related things only peripherally, so I do not represent the SMB team's point of view on this. Please do follow up with them.

<comment><USER17> It's only an extra round trip in the case of an unexpected EOF. File size is returned from SMB2_CREATE and so given the default of a RHW lease then (a) the lease can't be violated - if it is, then all bets are off as the server let someone modify your leased file outside the terms of the lease. Or (b) you know the file size, so a short read if you overlap the actual EOF is expected and you can plan for it.

A short read in the _middle_ of what you expect to be a continuous stream of bytes should be treated as some sort of server IO exception (which it is) and so an extra round trip to fetch the missing bytes returning 0, meaning EOF and something truncated or an error such as EIO meaning you got a hardware error isn't so onerous.

After all this is a very exceptional case. Both Steve's Linux cifsfs client and libsmbclient have been coded up around these semantics (re-fetching missing bytes to detect unexpected EOF or server error) and I'd argue this is correct client behaviour.

As I said, given the number of clients out there that have this bug we're going to have to fix it server-side anyway, but I'm surprised that this expected behavior wasn't specified and tested as part of a regression suite. It certainly is getting added to smbtorture.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER20> Is there any intention to optimize work done, rather than just the calling interface?

E.g., running an rsync if a 10m files hierarchy usually requires 10m synchronous stat calls. Using io-uring would make them asynchronous, but they could potentially be done more efficiently (e.g. convert file names to inodes in blocks of 20k, and then stat those 20k inodes in a batch).

That would require e.g. the VFS layer to support batch operations. But the io- uring would actually allow that without a user space interface change.

</comment><comment><USER21> Maybe I just missed this but can anyone tell me what kernel versions support io_uring. I ran the following test program on 4.19.0 and it is not supported:

        #include <stdio.h>
        #include <stdlib.h>
        #include <sys/utsname.h>
        #include <liburing.h>
        #include <liburing/io_uring.h>

        static const char *op_strs[] = {
          "IORING_OP_NOP",
          "IORING_OP_READV",
          "IORING_OP_WRITEV",
          "IORING_OP_FSYNC",
          "IORING_OP_READ_FIXED",
          "IORING_OP_WRITE_FIXED",
          "IORING_OP_POLL_ADD",
          "IORING_OP_POLL_REMOVE",
          "IORING_OP_SYNC_FILE_RANGE",
          "IORING_OP_SENDMSG",
          "IORING_OP_RECVMSG",
          "IORING_OP_TIMEOUT",
          "IORING_OP_TIMEOUT_REMOVE",
          "IORING_OP_ACCEPT",
          "IORING_OP_ASYNC_CANCEL",
          "IORING_OP_LINK_TIMEOUT",
          "IORING_OP_CONNECT",
          "IORING_OP_FALLOCATE",
          "IORING_OP_OPENAT",
          "IORING_OP_CLOSE",
          "IORING_OP_FILES_UPDATE",
          "IORING_OP_STATX",
          "IORING_OP_READ",
          "IORING_OP_WRITE",
          "IORING_OP_FADVISE",
          "IORING_OP_MADVISE",
          "IORING_OP_SEND",
          "IORING_OP_RECV",
          "IORING_OP_OPENAT2",
          "IORING_OP_EPOLL_CTL",
          "IORING_OP_SPLICE",
          "IORING_OP_PROVIDE_BUFFERS",
          "IORING_OP_REMOVE_BUFFERS",
        };

        int main() {
          struct utsname u;
          uname(&u);

          struct io_uring_probe *probe = io_uring_get_probe();
          if (!probe) {
            printf("Kernel %s does not support io_uring.\n", u.release);
            return 0;
          }

          printf("List of kernel %s's supported io_uring operations:\n", u.release);

          for (int i = 0; i < IORING_OP_LAST; i++ ) {
            const char *answer = io_uring_opcode_supported(probe, i) ? "yes" : "no";
            printf("%s: %s\n", op_strs[i], answer);
          }

          free(probe);
          return 0;
        }

<comment><USER22> If you have a clone of the Linux kernel source tree, you just have to look at the history of the include/uapi/linux/io_uring.h file. From a quick look here: everything up to IORING_OP_POLL_REMOVE came with Linux 5.1; IORING_OP_SYNC_FILE_RANGE was added in Linux 5.2; IORING_OP_SENDMSG and IORING_OP_RECVMSG came with Linux 5.3; IORING_OP_TIMEOUT with Linux 5.4; everything up to IORING_OP_CONNECT is in Linux 5.5; everything up to IORING_OP_EPOLL_CTL is in Linux 5.6; and the last three are going to be in Linux 5.7.

<comment><USER21> This article concurs. [https://lwn.net/Articles/810414/](https://lwn.net/Articles/810414/) io_uring was first added to the mainline Linux kernel in 5.1.

</comment></comment></comment><comment><USER23> Question: how does one detect socket push back using io_uring? For example, with libc "write/writev" for non-blocking socket would return less bytes than requested and allow code to poll for write readiness before writing more. This is quite useful to handle scenarios where there are impedance mismatches between processing speed and ability to send data over a network, e.g. processing needs to observe push back and handle it appropriately. Apologies: I posted this question to twitter before I read the redirect here.

</comment><comment><USER24> By coincidence I asked a few questions on the mailing list about io_uring this morning: [https://lore.kernel.org/io- uring/20200510080034.GI3888@redha...](https://lore.kernel.org/io- firstname.lastname@example.com/T/#u)

</comment><comment><USER25> Unfortunately I misread the title as "Lord of the Urine" and... was concerned.</comment>


Awesome Highlighter (YC summer 08) - <AUTHOR> http://www.techcrunch.com/2008/06/22/awesome-highlighter-isawesome/

<comment><USER3> Lets test it:

<http://awurl.com/pspxuq96279>

</comment><comment><USER4> Nice tool.

As I understand from Techcrunch, the revenue stream would come from selling their "awesome highlighter" product to media sites. The incentives for media sites to buy it?

\- nice feature for their users to share highlights

\- possibility to survey what their users find interesting

I believe there is some revenue potential in this. It just strikes me how simple business ideas can be. Once again, value is the key, not complexity.

<comment><USER5> User annotations help make the user feel they are part of the community/creative process. Walled gardens like NYTimes might be scared (remember ThirdVoice) to let the public annotate their content - <USER2> is a trojan horse to overlaying user annotations and co-opting their precious content. If I were the NYTimes, I'd be scared to sign any revenue-sharing contract unless it specifically prohibited later, more sophisticated annotations.

One of the most-popular sites in Japan is a clever site that lets anyone comment directly on a video, literally.

<http://en.wikipedia.org/wiki/Nico_Nico_Douga>

</comment></comment><comment><USER6> And on testing it, it does not even work. I tried yahoo.com and it keeps popping up this message box telling me there is a limit of 2000 characters, when I'm selecting small paragraphs. So YC funded a startup with an idea that has been tried many times over and with a product that just does not work? Hmm, somebodies gonna lose $10.000. Not saying who, but somebody....

<comment><USER7> Evaluating a first release like this is kinda silly. It's a road, not a destination. For a v1 (or a beta, which is what this really is), you're better off evaluating the road...

Regarding "tried many times over"... You mean like web search was before Google? Or MP3 players before the iPod? Or photosharing before Flickr?

I'm not saying I'm convinced this idea is a winner, but come on... Predicting failure for a startup is so laughably easy that I don't know why you'd bother, unless you just wanted to feel smug.

</comment></comment><comment><USER8> Congratulation to hooande. I remember when he mentioned it <http://news.ycombinator.com/item?id=174900>

Awesome Highlighter uses frames to view other websites and some websites prevent their pages to be viewed in someone else's frameset (if (top != self) top.location.href = 'example.com'). There is no way I know of to disable this and need to respect their wish not be viewed in frameset.

<comment><USER9> Actually they appear to be caching the pages, which is how they are able to detect what is highlighted (otherwise it would be cross domain scripting). So they could just have their caching script strip out that JS and problem solved.

I image that page caching could lead to problems with some sites though. I wanted to do something similar for a project I was working on but scrapped the idea because I couldn't get around the whole cross domain scripting issue, and caching just introduces a host of other problems.

</comment></comment><comment><USER6> I might be wrong, but I seem to remember seeing this same idea sometime in the web 1.0 bubble. It failed then, and since then, there have been several attempts to revive this very same business, and it fails each time. So why did YC go and fund a business that just does not seem to work?

<comment><USER10> The environment changes. Online video sharing was tried several times during Web 1.0, but didn't catch on until YouTube in 2005 - because in the meantime, broadband and cell-phone video and social networking had penetrated millions of households. Same with social bookmarking - anyone remember Links?

The fact that others have tried the same idea and failed is probably a bad indication, but it's not a hard rule that _this_ attempt will fail.

<comment><USER6> Yes, but get this: If someone had told me in 2000 that I could watch streaming video in good quality without any pauses and without needing any special computer or internet, I would have screamed excitedly: HOW? I would have WANTED it, but unfortunately, the reality of online video back then sucked bigtime.

If someone had told me in 2001 there was a way to easily annote webpages and share them with my friends, I'd have yawned. BORING!

Some concepts are just boring, and technology improvement does not change it. Those companies failed because people just don't have a need to mark pages. I can just copy and paste the text into emails.

<comment><USER10> But say you take Twitter as an example. If someone had told me back in 1995 that I could send 130-character text messages to all my friends over the web, I'd have said "So?". Now I've got a bunch of friends nagging me, "Are you on Twitter yet?" (I'm not, but I do get nagged about it fairly often.)

Technology doesn't just change what's possible, it changes what people want. The reason there's demand for something like Twitter is because now _everyone_ is online, nearly 24/7, and so the value of an online service that tells you what your friends are doing has gone up significantly.

You may be right - actually, from a numbers perspective, you probably are. But if I had a few million to invest and the chance to blow $15K on something like AwesomeHighlighter, I'd think there was _enough_ of a possibility of success to make it worthwhile. People are reading more online, and their reading is much more participatory. That's a different environment than in 99/00.

<comment><USER6> The only potential I see is in the tiny url service part of it. But I was only aware of this AFTER testing it out. It should be emphasised more:

Create a short url for a page - AND mark the parts you find interesting. Tiny url but with a marker.

</comment></comment></comment></comment></comment><comment><USER7> Check out:

<http://blog.linebuzz.com/>

They do the same thing without a plugin/bookmarklet.

I dunno about most folks, but I have to have some pretty serious pain and/or desire to add clutter to my browser.

</comment><comment><USER11> How does this differ significantly from Third Voice? <http://www.wired.com/techbiz/media/news/2001/04/42803>

</comment><comment><USER12> I love the name. It's simple, memorable, and instantly identifies what the site does. A very refreshing change from the plague of generic Web 2.0-ish non-word names.

</comment><comment><USER13> The bookmarklet does not work for me in Safari 3.1.1 The toolbar shows up and then disappears. Has anyone else tried it in Safari?

<comment><USER14> I have and the same thing happens for me.

</comment></comment><comment><USER15> They should definitely set cursor:pointer on the color selector for the ffox plugin.</comment>


Ask HN: Would this be legal? - <AUTHOR> suppose I have a chrome extension that allows anyone to monitor any website for new items, and when a new link appears, it opens a new tab to display the contents of the link. After the extension displays the contents of the link, it sends the html to my app, and my app processes it in some way and sends it back to the user in an email or something.
I imagine a site like Hacker News would have no problem with this. But if you take a site like Craig's List (canonical example) which has a litigious history, I could see them potentially having a problem. However, in this case I am not interacting with Craig's List in any way I am simply doing some processing on some text that is being sent to me.
EDIT: what does craigslist mean by "software expressly licensed by us" does this mean I can send them an email and get them to okay my software if its not something that is competitive with craigslist? <comment><AUTHOR> TOU: USE. You agree not to use or provide software (except for general purpose web browsers and email clients, or software expressly licensed by us) or services that interact or interoperate with CL, e.g. for downloading, uploading, posting, flagging, emailing, search, or mobile use. Robots, spiders, scripts, scrapers, crawlers, etc. are prohibited, as are misleading, unsolicited, unlawful, and/or spam postings/email. You agree not to collect users' personal and/or contact information ("PI").

However, in this case the software does not interoperate with CL in particular, it interoperates with all websites in general.

<comment><USER2> Regardless, it still interacts with CL in an autonomous way, right? Like a robot or crawler?

<comment><AUTHOR> well, it interacts with ALL websites the same, it attempts to identify a list of items, and makes an http request once for each item that comes up, and it does it client side where the chrome extension is installed.

So I suppose its at least sort of autonomous.</comment></comment></comment>


Hidden in plain sight: Brute-forcing Slack private files - <AUTHOR> https://www.ibuildings.nl/blog/2015/11/hidden-plain-sight-brute-forcing-slack-private-files <comment><USER2> Summary from my read of this: (the article does a great job of couching the process of exploiting this, as well as motivating why these numbers are too low, but here are the vulnerabilities...)

\- Slack chose to use a 6-hexadigit/24-bit "secret code" as the only/final code required to download "privately" shared files. That's _way_ too short; people have botnets almost that big, such that even aggressive IP-based rate- limiting wouldn't stand a chance.

They might have also made these fairly common mistakes (which served to compound the vulnerability):

\- Returning different/distinguishable error codes when the request matches correctly on some parts but not all. This allows attackers to guess each in turn.

\- Considering values such as the "file ID" to provide additional security/entropy, when in fact these IDs are generated _semi-sequentially_ , and thus a moderately-sophisticated attacker can narrow the search space dramatically.

\- Considering values such as the "filename" to provide more security/entropy; however, you can make no guarantees about the length or uniqueness of filenames, so you shouldn't consider that a security feature at all.

<comment><USER3> Also important to note it took Slack over a year to fix this issue often with long stretches of silence even with requests from the bug hunter.

<comment><USER4> Interestingly, I put the same bug in at HackerOne 9 months ago. It was closed as not applicable. So they had at least two independent reports of the same bug and failed to understand it, acknowledge it and then fix it.

Way to go slack.

If you have any critical data passing through slack, when you get owned, you won't be able to say say it wasn't entirely preventable.

</comment></comment></comment><comment><USER5> The correct answer for using URLs as capabilities (which is what a 'secret URL' really is: a capability to a resource, which can be handed out, copied &c.) is to use a 256-bit value as part of the URL. Thus, rather than '[http://example.invalid/TEAM-DOC-SHORT-RAND/'](http://example.invalid/TEAM- DOC-SHORT-RAND/') use '[http://example.invalid/w6uP8Tcg6K2QR905Rms8iXTlksL6OD1KOWBxT...](http://example.invalid/w6uP8Tcg6K2QR905Rms8iXTlksL6OD1KOWBxTK7wxPI'). If you're really paranoid, double the length. I guarantee it won't be guessed, in either case.

<comment><AUTHOR> Don't forget that it should ideally be cryptographically random. If the sequence is predictable (like based on an auto incrementing number or on time) then you might still be able to guess a 256-bit number.

<comment><USER5> > Don't forget that it should ideally be cryptographically random.

I thought that went without saying, nut yes, it must be cryptographically random (not ideally—it must be).

</comment></comment></comment><comment><USER6> > We apologize for the delayed reply. We track these issues via our internal > bug system, and only reply to the reporter once the bug is resolved > internally. We generally ignore messages asking for updates, as we receive a > high volume of these (even for non-issues).

This rationalization is illogical, which usually means someone is in conflict. From a logical standpoint, externally, it could be they are fixing something OR don't know about it OR don't care.

Given the conflicting rationalization, I'd say they didn't know about it and then made up an excuse instead of owning it.

<comment><AUTHOR> It probably means that they're not prioritising vulnerability reports. Which is their prerogative honestly, but it doesn't make researchers happy to work with you.

The biggest 'fault' here I think lies squarely with HackerOne. They should've enforced their own guidelines and given me the option to publish in their system after 180 days. But I _still_ don't have that option.

<comment><USER7> Great feedback, thanks.

The 180 day guidance you reference falls under a "Last Resort" clause when "... the Response Team [is] unable or unwilling to provide a disclosure timeline". (which, at first glance, might not have been the case here?)

These "Last Resort" scenarios have not yet been fully codified. As a safety precaution, the workflow is still initiated manually with support as these scenarios are extremely rare and littered with edge cases. We've been learning a lot from studying disclosures like this one and you can expect to see the "Last Resort" workflow codified in the product in the future.

Now that the report has been Resolved, you should see the normal disclosure options available. Please always feel free to send me a note if you have any questions or feedback on our disclosure workflows - especially if we don't support your preferred route.

</comment></comment></comment><comment><USER8> This reminds me of my experience with Imgur's private images.

A few years ago, I wrote a little js tool to browse random Imgur images by guessing their urls (i.imgur.com/<5-digit code>) until it found one that succeeded. It would add the found image to an infinite-scrolling page. It was kinda fun to browse, and a lot of people seemed to enjoy playing with it.

After a couple years, though, Imgur suddenly started blocking access to their images on my site. It turned out they were blocking based on the referrer header.

I emailed them asking what was up, and apparently they were attempting to ensure the privacy of public-url images by manually going after any tools like mine (if you google 'random imgur', you'll find dozens).

I didn't bother circumventing this, I didn't want to be a jerk just to prove a point. I _did_ try to point out that there were a number of ways to get around something as simple as a referrer block, but I don't think the customer support person I was dealing with was really interested in discussing the issue and I let it drop.

<comment><USER9> I had a similar experience, though I was on the other side. Under brute force login attack IT guy suggested I change login HTTP method from GET to POST (which is more appropriate anyway). While I agreed with him that this is better, I pointed out that this is very easy to circumvent. However he proved me wrong - the attacks stopped after that (and I am quite sure it is not because they gained access). Not all attackers are very determined...

</comment></comment><comment><USER10> What kills me is that this seems like such an unforced error. Just make 256-bit random tokens. They're private URLs. Who cares how ugly they are?

<comment><USER11> Not to mention making public file sharing explicit instead of implicit, or at least giving the option to exclude specific files from a public URL.

</comment></comment><comment><USER12> Damn, over a year to fix this? It's good that they did in the end, but the timeline is just crazy.

<comment><USER13> Maybe someone needed time to brute force all those tasty files hanging around ;)

</comment></comment><comment><USER14> The best part of the story actually comes at the end, back-and-forth messaging with slack about the bug report

<comment><USER15> definitely, I'm disappointed with slacks responses. We did a trial and have had some correspondence with their support team which has been excellent to date. So I assumed they were above some of this silicon valley elitism. I'm glad to see this kind of public disclosure. We have been a customer since that initial trial, we stopped using hipchat.

<comment><AUTHOR> To be fair, most of the bad correspondence was from 2014. Their new representative 'Leigh' appears to be doing excellent work. Also we're still happy users of Slack, I would just never trust them with secrets :-).

</comment></comment></comment><comment><USER16> If I were responsible for security at Slack, the thought of potentially leaking uploaded files like this would keep me up at night. Slack has gained such wide adoption-think of the things that people are sharing with their coworkers all day, every day. Someone with ill intent could have found so many valuable things.

</comment><comment><USER17> Simple how-to instructions:

      1. log into slack

      2. share a private file

      3. go to:

> [https://api.slack.com/web](https://api.slack.com/web)

      4. generate API token

      5. copy the link to your private file, and paste as plain text.

      6. a file id is in the link, somewhere

      7. try out the file id by visiting links like:

> > [https://slack.com/api/files.info?token=#secret!&file=???????...](https://slack.com/api/files.info?token=#secret!&file=?????????)?

      8. see also:

> > [https://api.slack.com/methods/files.info](https://api.slack.com/methods/files.info)

      9. in the JSON output you will CTRL+F to see an address like:

> [https://slack-files.com/#########-?????????-!!!!!!!!](https://slack- > files.com/#########-?????????-!!!!!!!!)!

      10. slackbot warns you, and only you, once and only once
          that someone found that link. this notification may
          get buried, or forgotten about. it is your only chance
          to revoke the public link.

      11. if you forget about that link, and it goes viral with
          millions of visits, lots of luck gentlemen!

</comment><comment><USER18> Do a google search for site:[https://s3.amazonaws.com/uploads.hipchat.com/](https://s3.amazonaws.com/uploads.hipchat.com/) , Atlassian does the same thing, and hasn't really addressed customer questions about it for a long time. [https://help.hipchat.com/forums/138883-suggestions- ideas/sug...](https://help.hipchat.com/forums/138883-suggestions- ideas/suggestions/5985543-sent-files-should-not-be-public-amazon-s3-links-t) is a long running Atlassian support ticket for this.

</comment><comment><USER19> Github does something similar, if you drag an image into the textarea in their issue tracker, it uploads the image to (I think) a public URL. I've considered what this could mean for teams with private projects who might e.g. attach screenshots with sensitive information.

Here's an example of an image uploaded via the GH issue tracker. Definitely public.

[https://cloud.githubusercontent.com/assets/95562/7319912/200...](https://cloud.githubusercontent.com/assets/95562/7319912/2004f03a-ea67-11e4-852f-f558af75b078.gif)

<comment><USER20> The good news is that github puts a uuid in the url, so its unguessable.

Slack, on the other hand, didn't have a big unguessable number... they had a very small number you could brute-force.

<comment><USER21> UUIDs are (usually) generated in a systematic fashion, so large parts of them are often possible to determine ahead of time.

<comment><USER22> Not in guid v4

<comment><USER23> Unfortunately that is not necessarily true, as nothing requires a UUIDv4 generator to use a cryptographically secure source of randomness.

</comment></comment></comment></comment></comment><comment><USER24> This should indeed inform potential users of additional security risks of using Slack, especially for communication of corporate confidential information. That said, this isn't a realistic threat to my own _social_ use of Slack.

</comment><comment><USER25> Similarly to using a hash function that is purposefully slow, wouldn't it be a good idea to introduce some artificial latency when responding to requests for urls like this?

</comment><comment><USER26> I see slack now has the option "Disable public file URL creation"

Now no-one can share files publicly in our companies slack channels</comment>


DuckDuckGo's privacy abuses – current, historic, and by proxy - <AUTHOR> https://dev.lemmy.ml/post/31321 <comment><USER2> I'm all for holding businesses to account but the arguments made on that site are pretty poor. For example, why should it matter that DDG uses Microsoft Outlook for their corporate emails? That has no baring on the DDG's ability to deliver a more privacy-focused search.

The expectation some people have about how some businesses or projects should be run is a little absurd.

<comment><USER3> Indeed. That and the wording of the text make it look more like a "hit piece" against DDG.

</comment></comment><comment><USER4> I don't know what to think of partnering with Bing and Yahoo or for using AWS. I don't see those as particularly bad for a search engine that's starting from scratch. Maybe we should expect more from DDG since it's no longer the new kid on the block and has money to reduce or eliminate such dependencies.

I always take care of cookies and tracking to the extent I can. So I don't know what else I can do to avoid using DDG or even Google (which I resort to for searches where DDG is inadequate).

 _> DDG is currently collecting users' operating systems and everything they highlight in the search results. (to verify this, simply hit F12 in your browser and select the "network" tab. Do a search with javascript enabled. Highlight some text on the screen. Mouseover the traffic rows and see that your highlighted text, operating system, and other details relating to geolocation are sent to DDG. Then change the query and submit. Notice that the previous query is being transmitted with the new query to link the queries together)_

FWIW, I couldn't find this behavior (or maybe I didn't figure out how to exactly check this) even when blocking nothing on DDG. Can someone confirm if this is true (or was true in the past but no longer true)?

<comment><USER5> Just tried, and it's true. Two REST calls to improving.duckduckgo.com are made. Can't tell if my OS and location is encoded in the query params, but there's at least a dozen or so populated fields.

</comment></comment><comment><USER6> I'm super skeptical of DuckDuckGo, but most of that is circumstantial and childish.

Apparently you're abusing privacy if you're using AWS from "privacy-abuser Amazon", or Outlook for your email, or selling T-shirts on a Cloudflare- powered website.

By that definition 99% of online businesses are abusing user privacy.

<comment><USER7> Hi. Why are you generally skeptical about DDG ?

</comment></comment><comment><USER7> Does someone have the knowledge / expertise to dig into these critics ? Well, the critics about partnerships and Cloudflare look like low level activism BS, but am interested in what is said under the section "Direct Privacy Abuse" in the article. Are these statements true and do we have to worry about that ?

</comment><comment> vertex-four I have no idea what this site is, but it won't load for me, at all - I just get a spinning circle for a few minutes. Ublock Origin doesn't claim to be blocking anything, so I don't know what's going on.

<comment><USER6> JavaScript-powered publication fear mongering about privacy, classic.

<comment><USER8> It's not a publication, just a post on some reddit clone

</comment></comment></comment><comment><USER9> Having seen parts of the original Names Database code my only objection to DuckDuckGo is that somehow it might have been contagious and infected DDG when Gabriel Weinberg launched the latter. This seems unlikely, but it was the most unfortunate Perl code I've seen in my career. On the other hand it _worked_, so perhaps I shouldn't judge it too harshly.

</comment><comment><USER10> Oof, that's not a pretty read for DDG. So now what!?

Often I look at people around me who are blissfully unaware of data mining etc. and feel envious. It's tiring.

</comment><comment><USER11> Does "uses the Bing API" mean that the search results are all fetched from Bing? As in DDG doesn't have it's own search algorithm? I'm surprised

<comment><USER6> Crawling the web is expensive, barrier to entry is very high.

Many alternative search engines claim to have built their own crawler, however their coverage is small and they have to fallback to Bing or Google.

You can actually check this btw. The results on Bing are identical to DuckDuckGo's in many cases, which are often inferior to those of Google. When different make sure the "location" parameters are the same and clear Bing's cookies, although local searches are absolutely terrible in Bing/DDG anyway. But they sometimes diverge on some queries, either because DuckDuckGo's cache is not up to date, or because it had relevant results in its own index.

But yes, if Bing closes its API, DuckDuckGo either switches to Google or it's dead in the water.</comment></comment>


BTCC to Cease China Trading as Media Warns Closures Could Continue - <AUTHOR> https://www.coindesk.com/btcc-to-cease-china-trading-as-media-warns-closures-could-continue/ <comment><USER2> All restrictions and regulations of bitcoin will result in distributed trustless solutions. We aren't that far from distributed, non-custodial, trustless crypto-crypto exchanges, and crypto-fiat are not that far behind.

More regulations will push the innovation outside whatever country passes the regulations.

Since bitcoin can be used in any crime that could also use cash, the idea that it can be used in crime doesn't justify these regulations-- unless you ban cash effectively the same way.</comment>


Ask HN: Does your project use Emacs lisp for build automation? - <AUTHOR>

I'm building a new appliance, linking BusyBox with Tor on top of a minimal GNU/Linux operating system. I'd like to try something different for the automated build infrastructure with this one and Emacs Lisp has been calling me lately.<p>Do you have a project written in C that uses automated build scripts written in Elisp? Guile might might be a runner-up but I need to get up-to-speed quickly on the infrastructure needed, so I'm hoping for Elisp for familiarity's sake. <comment><USER2> Not a build script per se, but I use elisp for some scripts which I manually invoke occasionally for source code transformations like generating tables and lists of enums from C source. I ought to make my build script invoke the elisp automatically.

</comment><comment><USER3> I think this would be awesome, but might be hard to make work. I don't know whether people run Elisp for purposes that aren't inside Emacs.

<comment><AUTHOR> Yes, that's what I'm finding is the mindset out there. And hang it if I'm not all the more compelled to do it with Elisp :-) Guile seems to be the logical next step (I've seen some stuff with a shell script container, using Gawk and Guile for the heavy lifting).</comment></comment>


Lowering reading levels in writing strangely increases views - <AUTHOR> https://medium.com/better-marketing/increase-your-article-views-25-by-writing-for-a-6th-grade-reading-level-6b862153d654 <comment><USER2> The pharma example,is interesting because the FDA requires that the dosing information printed on packaging for over the counter medicine be written for a fifth grade reading level.

It's not just the level of education but the clarity of instructions when the reader has a migraine or is worrying about their child.

This technique can be used as an antipattern: make the visitor's own "happy path" satisfy their reason for visiting g (say, purchase); stuff all your extra fees into places they'll skip over. The classic "fine print" technique.

<comment><AUTHOR> Interesting points

</comment></comment><comment><USER3> This isn't actually that strange.</comment>


JPMorgan will be the first major US bank to launch its own cryptocurrency - <AUTHOR> https://markets.businessinsider.com/currencies/news/jpmorgan-cryptocurrency-launch-jpm-coin-a-first-for-a-major-us-bank-2019-2-1027953761 <comment><USER2> It'd be cool to see a whitepaper on this.</comment>


Pre/post Spectre iPhone 7 benchmark (2% performance increase) - <AUTHOR> https://browser.geekbench.com/v4/cpu/compare/6303880?baseline=6303560 <comment><AUTHOR> And here's one of my iPad Pro 10.5": [https://browser.geekbench.com/v4/cpu/compare/6304146?baselin...](https://browser.geekbench.com/v4/cpu/compare/6304146?baseline=6303715) (0.9% increase in performance).</comment>


Do What You Love - <AUTHOR> http://blogs.valvesoftware.com/abrash/do-what-you-love/ <comment><USER2> I always hated that advice "Do what you Love". It's a fucking terrible advice. I know people mean well when they give that advice, but its a downright terrible advice for the vast majority of people.

Let me point out that this is an article written by a guy that ended up "loving" programming. I know that there is a lot of programmers here, (myself included) but let's face the reality here: only weirdos love programming. I say this because I'm a weirdo and I love programming, and every friend I know that loves programming is more or less the.. eccentric type. We just happen to be very fortunate in that we happen to love something that guarantees a middle class income and plenty of opportunities.

Most people are normal and love normal things. They like food, they like music, they like art, they like sex, they like sports, they like adventure. If you tell normal people to do what they love, you are practically dooming them to shit careers. It's all about supply and demand. Most people love the same things that other people love (except programmers, who are weirdos), and there just isn't enough jobs that normal people will love. Maybe 1% of normal people will end up in their dream job, and the other 99% end up on the hamster wheel chasing what they love.

<comment><USER3> I think your complaint goes away if you complete the sentence: Do what you love _to create_

Most people live read-only lives. They never create music, food, or art. They merely consume them.

But if you _love_ creating music, food, or art you absolutely can make a career out of it. Of course, a middling chef or musician won't make it very far, but neither will a middling programmer really (though they may at least stay employed).

<comment><USER4> I don't think his complaint goes away at all.

For one thing, in my experience, most people _don't_ love to create. Most people aren't artists, programmers, writers, chefs, or carpenters because they wake up in the morning and look forward to it. My overwhelming impression from the vast majority of people I've met in my life indicates that _creating things_ is simply not a driving force. If given a complete lack of want, I'm convinced the majority of the world's population will choose to do nothing creative with it, and spend it socializing, fucking, and eating.

Scarcity is pretty much the only reason their butt finds their way to the desk every morning.

Not that there's anything necessarily wrong with that.

Secondly, even for people who have strong passions in a particular craft, there's no accounting for supply and demand. We are _incredibly_ fortunate that our passions lie in a field that is _severely_ under-supplied, allowing us to enjoy incredible compensation and job security in an era where so few have just that. And our situation is highly temporary, I assure you. The ability for your passion to put a roof over your head and provide the basic necessities of life is... questionable.

So while I'm pulling in a 6-figure salary, have insane perks at the office, flexible hours, live in an amazing city, and have the money and time to pursue my dreams, I'm not going to stand on a soapbox and pontificate to everyone how, _if they had only followed their passion_ , they could be happy too.

The reality is that we are the lucky few. For a vast portion of the population following their passions is a one-way ticket to getting evicted and digging through garbage cans.

<comment><USER5> _If given a complete lack of want, I'm convinced the majority of the world's population will choose to do nothing creative with it, and spend it socializing, fucking, and eating._

The reality TV show _Big Brother_ is strong evidence in favor of your argument.

It's an artificial environment where people are aggressively prevented from creating anything. You can't even smuggle a pencil and a piece of paper into the Big Brother house -- if you wanted to write or draw something, you'd have to do it with your own blood on the walls, I guess... Once you're in there, your life for the next N months will consist solely of socializing, fucking and eating.

Amazingly, there are millions of people who find this prospect so exciting that they apply for the show year after year.

<comment><USER6> I don't think we should draw general conclusions about humanity based on reality show contestants. The draw to participate in Big Brother is that the house is full of TV cameras. Which, ironically, could actually be described as creating something.

</comment></comment></comment></comment></comment><comment><USER7> Unfortunately, you only get to write these sorts of posts when what you love is economically in demand.

<comment><USER8> More like people only want to read them when what you love is economically in demand. These always remind me to go back and reread the sections on survivorship bias in fooled by randomness. I'm not sure if that's ironic or not.

</comment></comment><comment><USER9> This:

"In general, try things that seem worthwhile, set goals and work hard to achieve them, and see where that leads and how you respond. It'll be clear when something becomes compelling"

Everybody says "do what you love". Few say how to find what you love. Fewer still say how to manage to work as hard as you should to find what you love.

It's kind of a circular problem, when you love something you'll naturally work very hard on it, when you are still searching what it is you love, it's really hard to get the energy to work as hard as you should to find the elusive loved mission.

<comment><USER10> As a recurring depressive, I can't help but think, "How the hell do people live day to day not knowing at least one thing they absolutely love?"

If not for my own things that I intensely can dig into, I would've offed myself a long time ago.

<comment><USER9> It's amazing how different how life is lived and understood between depressive and non depressive people.

I personally have a really hard time even imagining depression, and I ask myself: "how can it be so hard to just pull yourself together?"

I can rationally understand it is really hard, but I can't imagine it.

<comment><USER11> Imagine having nothing that you love to do, and imagine the things that you previously loved to do are now laborious and draining, no matter the positive things you tell yourself about them. Think of something you really enjoy doing, and really look forward to, now imagine replacing the positive anticipation with dread instead. Imagine knowing you should be getting positive feedback from some accomplishment or activity, but all that your brain and emotions register is "meh." It's not that you don't try to pull it together and keep slogging forward every day, achieving things and smiling at people, it's just that none of your efforts ever seem to pay off emotionally. It's not that you don't recognize and routinely count your blessings, but rather the very act becomes negative as you lament not feeling much of anything for them. I understand the sentiment that it's hard to imagine, I hope this description may help.

</comment></comment></comment></comment><comment><USER12> I think it's important to remember that "Do What You Love" may not lead to _financial_ success. It will only lead to you doing more of what you love, which may or may not ever pay well.

(I'm a stay-at-home dad, and I love it. But it doesn't generate a lot of cash.)

</comment><comment><USER13> I get to quote from Gladiator, yeah!

In the movie, Cicero says: "Sometimes I do what I want to do. The rest of the time, I do what I have to."

I love building things and programming, but I love it when it is a process of self-accomplishment, driven by a deep desire to create, discover and learn.

In my job environment, however, these values are irrelevant and what I love to do otherwise becomes what others trample and waste.

So, yes, do what you love to do when you can and do what you have to do the rest of the time. Unless you don't care about living under a bridge.

</comment><comment><USER14> _Every_ time I've tried to push my career in some direction, I've been left unhappy and frustrated. Whenever I've held on and enjoyed the ride, I've found myself in new and exciting territory. I have no idea what tomorrow will bring, but I know enough to stop trying to force what I think it should be; instead, I'll watch and see what happens and make the best of it.

</comment><comment><USER15> This was the highlight for me:

"Perhaps the most useful way to figure out what you really want to do is to observe what you actually choose to do."

In other words, what do you do in your free time?

<comment><USER16> It seems so weird to me that we've gotten to a point where people don't think they are what they do.

<comment><USER17> Well, they aren't. They just cognitively distance themselves from it because it sucks.

And I say this as someone with an achievement neurosis. I fight this perception of myself and others because it ultimately does not benefit me -- I'd be working hard either way. No need to pile my identity onto it.

</comment></comment></comment><comment><USER18> Do what you love doesn't pay the bills.

If I did what I loved, I'd be in the garden or knee deep in solder.

I do what I have to do, so I'm chained in front of Visual Studio.

<comment><USER19> There is money in gardening - I have friends that are very successful at it, and are still hands on, picking and planting the plants - while making more than enough to live a happy life.

My father loves drawing and making stuff, he is a pretty damn successful stage designer.

You have to love what you are doing AND be good at it AND good at selling yourself. But if you can crack that you really can love getting up each day. I know I do, and I know most of my friends and family do.

My richest friend works 60 hour weeks to have a great car parked outside his office and to support a stunning home he never 'lives' in - and he will tell anyone that no only does he hate his job, he isn't happy. But he really thinks having more (holidays, watches, suits, etc.) will make him happy, but they haven't yet...

</comment></comment><comment><USER20> Do garbage men and sewage workers love what they do? I doubt it. Are their jobs absolutely vital to the very existence of modern civilization? Absolutely.

"Do what you love" seems to fail if you wish it universal as a categorical imperative.

</comment><comment><USER21> On this same topic, Mike Peters, pulitzer prize winning cartoonist, gave a brilliant commencement speech at Washington University in St. Louis this May.

I admit though, I consider myself one of the lucky few (and probably stand amongst many of you in this category) for whom doing what I love coincides with doing what actually makes me "successful" by our society's standards.

I loved this line though, perfectly described my college experience and discovery of programming: "I was good at lots of other things I did academically, but they all felt like work."

</comment><comment><USER22> Going along with grlthgn, The bigger problem with 'Doing what you love' is most times, we are not aware of what we love. It is just impossible to find what you love. Sometimes the thing we love is short lived. This leads us to the next question, do everyone in the world have some this life-long activity they love? That question has never been answered.

<comment><USER17> It's not impossible, but it does require introspection, and a willingness to change things up. The combination of qualities excludes many people - you can have introspection without action, but it can end up in depression.

It's also partially generational, too. Later generations aren't as tied to their jobs because they grew up in a world where, collectively, employers didn't take quite as good care of employees.

</comment></comment><comment><USER23> Unrelated but that background with the pale hues makes it tough for me to read. Maybe Im just getting old.

</comment><comment><USER24> I love when people write these kinds of things, and include all the details. Makes it so much more memorable.

Good read.

</comment><comment><USER25> "Do what you love, but please do get your PhD first. You do love academia, right? Right???"

</comment><comment><USER26> I'm really glad that the guy who picks up my garbage doesn't do what he loves full time.

</comment><comment><USER27> In a Stanford lecture, Tina Seelig said something along the lines of: find where your interests, what your good at, and the market (ie someone will pay you) intersects and do that. I've always liked that advice.

</comment><comment><USER28> Very well said...it is not the destination but journey that matters most...

</comment><comment><USER29> Do opportunities.

That's all about and if these opportunities are about your passion even better. Otherwise you wait for the perfect idea or stuck with a crappy thing for years only because it's you passion.

</comment><comment><USER30> Personally, I prefer those moments in life when everything seems to strike a nice balance. I find they give me insight into what I love to do.

And yes, it generally involves creating something.

</comment><comment><USER31> Side note, but can some one explain or link to something that explains xor- based animation? I couldn't find much in a quick Google search.

<comment><USER32> When the display only has black and white pixels, you can XOR the a "sprite", or image to the screen to draw it, and XOR the same sprite in to the same location again to undraw it. The advantages are that you don't have to redraw the entire screen this way each frame, images can overlap, and you can draw your image on a black background or a white background.

For animation, you pre-XOR the sprites together so that a single XOR both erases the old image, and draws the new one, in a single blit.

<http://z80-heaven.wikidot.com/sprite-routines> has some sample code and perhaps a better explanation. <http://tibasicdev.wikidot.com/68k:sprites> has some sample pictures.</comment></comment>


Gamesibeat – gaming website - <AUTHOR> https://www.gamesibeat.com/ <comment><AUTHOR> Hello All,

I made this site for fun as a personal project. The premise is that I wanted a fun way to keep track of the games I played. I realize there are other apps. I am just looking for feedback regarding landing page, navigation, features and overall usability. Any constructive advice will be appreciated.

Thanks, gib

<comment><USER2> the landing page is just a registration form, it gives me no idea what the site is about or why i should give you my email address to register

<comment><AUTHOR> So I did a lot of research and many pages had a similar landing page. What do you think is a good alternative? Maybe a video up top describing the site?</comment></comment></comment>