When an SQL Database Makes a Great Pub/Sub - <AUTHOR> https://threedots.tech/post/when-sql-database-makes-great-pub-sub/ <comment><USER2> There's a perspective that the transaction log of a typical RDBMS is the canonical form and the rows & tables merely the event-sourced projection. After all, if you replay the former, you should always get exactly the same in the latter.

It's curious that over those projections, we then build event stores for CQRS/ES systems with their own projections mediated by application code.

Let's also mention the journaled filesystem on which the database logs reside. And the log structure that your SSD is using internally to balance writes.

It's been a long time since we wrote an application event stream linearly straight to media, and although I appreciate the separate concerns that each of these layers addresses, I'd probably struggle to justify them from first principles to even a slightly more Socratic version of myself.

<comment><USER3> Indeed, I've also puzzled over this a fair bit. Its almost as if we are lacking a lower-level interface to the transaction log that enables one to push events without going via the higher-level representation of tables etc. However, the implementation details are somewhat beyond me :-) Postgres' Bottled Water [1] was what made me think about this. I mean, why bother with exporting to Kafka at all, and instead just use the Postgres transaction log directly?

[1] [https://github.com/confluentinc/bottledwater- pg](https://github.com/confluentinc/bottledwater-pg)

<comment><USER4> It's much easier to scale kafka than a relational db. There's also an advantage to offloading the read load to another system instead of hitting the db for that, too.

<comment><USER3> Before I proceed, let me just state that I have no particular knowledge of either RDBMS' internals or Kafka, so this is nothing but my own amateur musings on the subject.

Now, I wholeheartedly agree that Kafka is more scalable, but I think the key point here is that there is no particular law of nature as to why that is the case. It may just be an historical accident of how RDBMS - and PosgreSQL in particular - have evolved. Further: many of the properties of Kafka are in fact also desirable properties for the PostgreSQL transaction log.

My take on the inopinatus observation, together with the Samza article [1] mentioned on this discussion, are as follows. You can think of Postgres as two "products" (bounded contexts, if you like):

\- a stream-based, possibly replicated, transaction log;

\- a projection of that transaction log into relational calculus, plus all of the associated machinery.

Thus far we never had the need to think of these as clearly separate "products", but Kafka makes it obvious that they are. In truth, the amount of tools processing WAL outside of PostgreSQL were already hinting in this direction; Kafka just made it obvious.

From this perspective, it seems a tad expensive to take the original transaction log, convert it to a RDBMS representation, then convert it to events and, in some cases, then store it as an event stream in Kafka. It would be much more efficient to simply use the original transaction log directly - and this is why, to me, even Debezium [2] / Bottled Water [3] appear to be one layer too many. To the best of my understanding, this line of reasoning is also line with the observations in the Samza article [1]. Where I believe I differ from the article is in thinking that the RDBMS representation also adds a lot of value to applications - I see both having a role (e.g. streaming vs batch processing sort of thing). I think this would derail the present discussion too much, so I won't go in to it.

In conclusion: to the untrained eye, it seems that the right thing to do is to extract the transaction log out of PostgreSQL and make it as scalable as Kafka. Then, allow for it to log "things" which are not necessarily "projectable" into the relational plane. PostgreSQL then becomes just a client of the transaction log, together with other "kinds" of clients. I suspect that this is what will ultimately happen, but the engineering work required will probably span a decade or more.

My 2 Angolan Kwanzas, at any rate.

[1] [https://www.confluent.io/blog/turning-the-database-inside- ou...](https://www.confluent.io/blog/turning-the-database-inside-out-with- apache-samza/)

[2] [https://debezium.io/](https://debezium.io/)

[3] [https://github.com/confluentinc/bottledwater- pg](https://github.com/confluentinc/bottledwater-pg)

<comment><USER4> Kafka can scale and distribute individual streams across the cluster. That is, the entire "database" is distributed across nodes. With postgres, your unit of scalability is the entire database. You can't natively have some tables on one node and some tables on another node, for example.

Kafka is also just more optimized for what it does. Postgres is a superset of what kafka does, so kafka is unsurprisingly better able to optimize for its usecase. It has a zero-copy protocol that can shuttle data to/from disk to/from the network without bringing it into memory (using the sendfile syscall). It doesn't wait for disk writes when doing writes, because it achieves durability via replication.

Also, don't forget the things you'd have to do when implementing consumers. How will you load balance streams between consumers? Meaning, if you have a stream and you want multiple consumers to burn it down at a time, how can you make sure they aren't duplicating work and they can handle the consumer group growing/shrinking? How will you handle checkpointing where each consumer tracks what they've done so far? What about streams' data rolling off?

All of this is doable with pg, but you'd have to implement it yourself. With kafka and its client drivers, this is handled for you.

<comment><USER3> As I said, I haven't given a lot of thought about this so please take my opinion with a grain of salt - but I believe that once you split the log out of PostgreSQL, a lot of functionality of this ilk could start to be considered. When/if added, I think it would make for a stronger PostgreSQL in the end. However, I do understand this is an insanely hard amount of work. In a way, it bears some similarities to splitting GTK out of GIMP, for instance; extremely difficult thing to do but ultimately it turned out to be a massive win for both projects. This would be even harder, but ultimately, greatly advantageous.

</comment></comment></comment></comment></comment></comment><comment><USER5> Really, I wouldn't teach junior developers that it's ok to use a database table when a queue is needed. Sure, you can get away with this and there are cases when it's all you need. But I've been one of those juniors who forgot to limit the query, who didn't have enough indices, who tried to order all records by date and had full table scans everywhere, who implemented the worker with a cron job and didn't synchronize this with a lock.

It might work, but it's not the general case and you might spend more time to debug your table then to write the code to use a real queue.

And I've also seen people build their own queueing engine for a few hundred tasks per day. Why don't they just choose one of the very good open source solutions?

<comment><USER6> "Because someone might add bugs to the code, or do it wrong" isn't a reason not to take a particular approach.

<comment><USER7> Of course it is. When one approach is far more likely to introduce bugs, complicated interactions, and be more difficult to maintain, it is absolutely a reason to not take a particular approach. It's one of factors you need to consider in everyday software engineering.

If you don't take this into consideration then you're detracting from the business to satisfy another need.

</comment></comment></comment><comment><USER8> The "database as message queue" pattern is quite common and often considered to be an antipattern, which I tend to agree with but I don't have that strong of a position on it myself. I've certainly used this pattern for expediency, but that was before we had all the messaging solutions we do today. [http://mikehadlow.blogspot.com/2012/04/database-as-queue- ant...](http://mikehadlow.blogspot.com/2012/04/database-as-queue-anti- pattern.html) has some good points.

<comment><USER9> A lot has changed in the 7 years since that was written.

Polling isn't a huge issue to begin with, and is mitigated with LISTEN/NOTIFY (on certain DBs). Inserts with indexes are not a performance problem at the scale of most applications. A separate messaging service won't prevent you from building a "hugely coupled monster".

Personally, I almost always start with the database as a queue. The operational overhead of running, updating, and monitoring another entire service is non trivial. If the messaging rate exceeds the database's capabilities in the future, I'll migrate then.

<comment><USER10> If you need just one queue yes. If you have lots of queues it's worth investing in a queue service of some sort and there are many of them out there which is a good thing but could turn into a bad thing quickly. In the past I worked at a place that had 3 different queueing services implemented by different developers and it became a pain to manage them or to even know what was on the queues.

</comment></comment></comment><comment><USER11> I've always considered message-queues as a close cousin (if not sibling) of databases. Arguably performing the same function with different foci. Pub/sub focusing on the "oplog". DBMS focusing on "state".

(Blockchain another "oplog" that ends up caring a lot about state eventually).

It's no wonder you can use them interchangeably in many common base cases.

</comment><comment><USER12> Seems like they fall into the same pit as many does: using primary keys with autoincrement as offset. This leads to skipping messages because there is no guarantees that primary keys will be available in monotonic order. Because, you know, transactions.

<comment> Fire-Dragon-DoL Can you expand a bit on this? From my understanding, autoincrement keys ca mn have gaps, but are always increasing. Sometimes a message might arrive "late", so you get a 3,then a 2. This problem cannot really be solved without giant locks that are not ideal. As far as I'm aware, all messaging systems are subject to this problem.

Messages will never arrive, arrive out of order and I don't remember the third one right now (messages will arrive late?)

<comment><USER13> Databases such as postgresql will effectively issue a buffer of keys to each connection, meaning in some circumstances the sequences will not be monotonic with respect to time. Also that usually long running transactions will use the timestamp the transaction was opened, regardless of how many seconds have passed between then and when the statement is executed.

<comment> Fire-Dragon-DoL Very interesting details, thanks. So the alternative is have inconsistency, or "giant locks". One is not performant, the other is inconsistent.

Tough choice, interesting nevertheless

</comment></comment></comment></comment><comment><USER14> Perhaps it's not so much about pub/sub, but about store-and-forward.

When the "forward" part of "store-and-forward" is most important then Kafka is a fine solution.

However, when the "store" part - for example you want to be able to stream historical data again, or interact with the data in different ways - is most important I have recommended HBase (+ Phoenix) as a better solution in the past.

</comment><comment><USER15> For anyone just looking for 'plug and play' web socket pub/sub functionality, I have been developing something that provides the functionality for PostgreSQL: [https://github.com/supabase/realtime](https://github.com/supabase/realtime)

It's an Elixir server (Phoenix) that allows you to listen to changes in your database via websockets. Basically the Phoenix server listens to PostgreSQL's replication functionality, converts the byte stream into JSON, and then broadcasts over websockets. The beauty of listening to the replication functionality is that you can make changes to your database from anywhere - your api, directly in the DB, via a console etc - and you will still receive the changes via websockets.

The article suggests Postgres' native LISTEN/NOTIFY functionality. I tried that originally and found that NOTIFY payloads have a limit of 8000 bytes, as well a few other inconveniences.

It's still in very early stages, although I am using it in production at my company and will work on it full time starting Jan.

<comment><USER16> One way to get around the 8k NOTIFY limit is to only use the capability to notify only. It would them be incumbent on the client to go fetch the data from a table somewhere. I ran into a similar limitation with SQL Server 2005 years ago and used this approach with great success.

</comment></comment><comment><USER3> MessageDB was doing the rounds in reddit the other day [1]. Looks interesting for simple use cases...

[1] [https://www.reddit.com/r/PostgreSQL/comments/ebu6nh/message_...](https://www.reddit.com/r/PostgreSQL/comments/ebu6nh/message_db_event_store_and_message_store_for/)

</comment><comment><USER17> That's basically the same pattern as the "outbox pattern", e.g. listed in Chris Richardson's pattern of microservices patterns.

An alternative implementation is provided by Debezium [1], a general solution for change data capture for MySQL, Postgres, MongoDB, SQL Server and others, based on top of Apache Kafka (but can also be used with Pulsar and others).

There's support for outbox coming as part of Debezium out of the box [2].

Disclaimer: I'm working on Debezium.

[1] [https://debezium.io/](https://debezium.io/) [2] [https://debezium.io/documentation/reference/1.0/configuratio...](https://debezium.io/documentation/reference/1.0/configuration/outbox- event-router.html)

</comment><comment><USER18> If anyone is using Elixir, Oban[0] is a job processor that uses PostgreSQL for its back-end and state management.

It's incredibly well written and I am using it in a project.

[0]: [https://github.com/sorentwo/oban](https://github.com/sorentwo/oban)

</comment><comment><USER19> One fun open source software I've played with, that I don't think many have heard of, is Deepstream.io. It attempts to be a batteries included real-time web server that works with websockets, and can function as pub/sub server and client. It has a connector for using PostgreSQL as the database. The frontend JavaScript library is really easy to get working.

[https://deepstream.io/tutorials/concepts/what-is- deepstream/](https://deepstream.io/tutorials/concepts/what-is-deepstream/)

[https://github.com/deepstreamIO/deepstream.io](https://github.com/deepstreamIO/deepstream.io)

(I'm not affiliated with the project.)

<comment><USER20> Thanks, this does really look interesting. I'd like to find something generic, lightweight to replace Kafka or Pulsar. Not sure this could be it, but it looks like it'd be worth having a look at...

</comment></comment><comment><USER21> This could slightly out of context.

I'm working on a module that send notifications to a user when an alert is generated. I have PostGreSQL as the database and NodeJS is the handler and for connection pooling. Are there any good pub/sub tools that I can use. Thanks in advance.

<comment><USER22> How about simply having an after insert trigger on an alerts table that calls notify, and then you listen for that in node? It's a simple setup with less moving parts and could probably get you a long way...

<comment><USER21> Thanks, this will probably work for me. However, if the inserts are higher, I don't want to get notified that frequently. How would I add a periodic alerts to this? Thank you!

<comment><USER22> You could use a column/table to track notifications. Eg. have an alert_sent_at column on the alerts table. Then when the node service receives the notify on an insert you can defer based on any logic you need, and send notifications when needed by a query that fetches alerts with alert_sent_at = null.

<comment><USER21> Thank you! Thank you so much :)

<comment><USER22> You're welcome. To ensure you're not sending alerts twice either select and update in a transaction or use a CTE like:

      with alert as (
        select alert_id from alerts
        where alert_sent_at is null
      )

      update alerts set
        alert_sent_at = now()
      from alert
      returning *

</comment></comment></comment></comment></comment></comment><comment><USER23> Meteor provides pub/sub with a MySQL backend using the atmosphere package vlasky:mysql.

It works by following the MySQL binary log and triggering a reactive query based on event conditions specified by the programmer, e.g. a change in a field.

[https://atmospherejs.com/vlasky/mysql](https://atmospherejs.com/vlasky/mysql)

</comment><comment><USER24> Oracle has a very advanced and flexible system for this. It is called Advanced Queueing.

</comment><comment><USER25> Ever use google's firebase? While not SQL -- I've always felt `tis a nice solution to persistence+async...</comment>


You're 'Raised Right' When You're Poor - <AUTHOR> https://melmagazine.com/en-us/story/youre-actually-raised-right-when-youre-poor <comment><USER2> Having grown up poor myself, I can attest to this. My mother was preoccupied with looking respectable. It was one thing to be poor, she would say, and another thing entirely to be poor white trash.</comment>


GitHub announcements: Marketplace, Apps and GraphQL API - <AUTHOR> https://github.com/blog/2359-introducing-github-marketplace-and-more-tools-to-customize-your-workflow <comment><USER2> Hm, I like Github but I don't like this. This is a great move, but we already have a code hosting monoculture, and this subtly piggy-backs on it towards lock-in for tooling. Github knows orgs have inertia.

Sorry for the slippery slopeism. I'm just imagining a future where shipping an app without a middleman moves to inconvenient, then annoying, then difficult, then impossible. Because this is exactly how we lost the ability to send email without paying the middleman privacy tax.

<comment><USER3> > Because this is exactly how we lost the ability to send email without paying > the middleman privacy tax.

You want to see a tax? How does $72/user/year for code coverage tooling sound?

<comment><USER4> What regime forces you to use paid code coverage tooling? What percentage is $72 of your annual per-developer budget?

</comment></comment></comment><comment><USER5> The thing I'd like to see Github focus on with their API is granular permissions. There are many SASS integrations that I am just not comfortable with granting access to because I don't want to have any more than necessary 3rd parties having access to repositories. Please correct me if I'm wrong, but every time I grant an application access to my Github account, it automatically has access to all repositories I have access to. I know there are ways that organizations can set permissions such that they can approve app integrations. Still, there are private repositories I have that I would prefer to not have 3rd party integrations be able to access. With 3rd party SASS providers having access to private repositories, it would be trivial for code to be stolen or sold to competitors. If they wanted to be real nasty they could rebase and force push, delete branches, tags, inject malicious code. There are so many potential ways that providing full access to everything you have access to in Github could be exploited.

I would love to see Github add permissions to apps requesting OAuth permissions be able to restrict access to the repositories the 3rd party app has access to. It would also be nice to be able to revoke individual permissions.

[Edit] fixed some grammatical errors.

<comment><USER6> +1 this a thousand times. The only way i have found to effectively lock things down is using an org and creating a new Github user just for third-party integrations.

<comment><USER7> Bitbucket is fast, and I believe focuses on repository permissions. Have you tried them?

<comment><USER6> Yes, though it has been a year or two. However it's more of a personal preference / fits within my normal work routine to use GH. I have a number of OS repos that, for community reasons, need to stay on GH, then most of my normal work is on GH which is generally where the concerns come in from a security standpoint.

</comment></comment></comment></comment><comment><USER8> I just got an advertising email about this from GitHub, even though I've never subscribed to GitHub advertising mail. I just checked my email settings and it looks like GitHub has added a new option (defaulting to being "on" of course) that auto-subscribes you to spam.

Surely the unsubscription laws don't allow you to retroactively add more options that are defaulted to being on that subscribe you to things you were never subscribed to in the first place? Seems like an easy way to spam people without having any repercussions.

For those who want to disable this anti-feature, look at the bottom of [https://github.com/settings/emails](https://github.com/settings/emails).

<comment><USER9> I usually give aliases for such accounts because of this (and because of data breaches). Send me spam and I don't have to put up with your UI to unsubscribe, I just delete mail alias. Of course, that doesn't work if I want to receive any mail from you...

<comment><USER8> What email service do you use? I've been trying to figure out where to switch to from "the big G" and it's not clear to me what host is good.

Personally a host that has a mail client which is free software would be the best deal. :P

</comment></comment></comment><comment><USER10> After trying new GitHub Desktop Beta mentioned in the article a week ago: it's still nowhere near compete enough to work:

\- You can only stage all changes in a file - so you can't discard 'console.log('Wooo'); debugger;' when committing, you have to remove those in your editor first, then commit.

\- you can't reverse commits. Need to roll HEAD back to a previous commit, push those fixes, then resume what you were working on? You can't do it.

\- still no graphical interactive rebase to merge and squash and reorder commits (the main reason people put up with sourcetree).

It feel fast and responsive though.

Edit: downvotes seem odd. If you think this is offtopic, the GitHub Desktop Beta is announced in the article.

<comment><USER11> > You can only stage all changes in a file

If I'm understanding you correctly, this is wrong. You can add individual lines in a file to your commit. That's one of the main uses I have for the app: it's easier to split up file changes between commits than on the command line. See this screenshot: [http://imgur.com/a/4CQG7](http://imgur.com/a/4CQG7) You have to click on the line numbers to add/remove individual lines.

<comment><USER12> Not a very obvious UI!

<comment><USER13> UI is same as Tower. Doesn't seem that far off for me.

<comment><USER14> No, Tower works differently (unless it has changed in the few months since I used it). In Tower, you can select lines by clicking line numbers, but then you must click "Stage" to actually stage those lines. Then your selection is cleared and those lines are moved to a separate "Staged" tab for that file. Whereas with GitHub Desktop, just selecting the lines marks them as lines to be staged. (GitHub Desktop doesn't actually run `git add` on those lines until the moment you click "Commit".)

</comment></comment></comment></comment></comment><comment><USER15> I've recently been exploring GraphQL. It's an excellent way to design and maintain Web APIs imo. GitHub is a great resource for learning how GraphQL works. I've been using the GraphiQL tool to experiment with it. However, there is a big issue with the GraphQL implementation in that requests using the POST method work, but the mirror GET requests do not return the same results. For example, see the response for the simple query

"query { viewer { login }}"

My understanding is that this request should return the same result, whether or not it is a POST or GET request ([http://graphql.org/learn/serving-over- http/#get-request](http://graphql.org/learn/serving-over-http/#get-request)). Could someone provide some clarity on this?

<comment><USER16> Perhaps [https://developer.github.com/v4/guides/forming- calls/#commun...](https://developer.github.com/v4/guides/forming- calls/#communicating-with-graphql) is relevant:

> In REST, HTTP verbs determine the operation performed. In GraphQL, you'll > provide a JSON-encoded body whether you're performing a query or a mutation, > so the HTTP verb is POST. The exception is an introspection query, which is > a simple GET to the endpoint.

<comment><USER15> Thanks. I'm not sure that this conforms to GraphQL best practices. [http://graphql.org/learn/serving-over- http/](http://graphql.org/learn/serving-over-http/) It seems GET methods should be supported for more than just introspection queries.

</comment></comment></comment><comment><USER17> How about communities, forums, boards? When I see an interesting project I'd like to talk to the developer without cluttering the Issues section.

It could be the entry point of every repo, moderated by the owner of course (something like php docs where people can post interesting stuff related to the page at the end). And it would make Github more social.

<comment><USER18> Why do you think you'd be cluttering the issues section? I wouldn't let that (your) perception stop you. Just open the issue if you have something to say.

<comment><USER19> Please don't. Issues are for bug reports and are not a support forum. If the repo maintainer asks in the README that usage questions should first be asked on Gitter, Slack, or Stack Overflow, please respect that request!

<comment><USER20> Where does a "hey, I have this idea, could send a PR if you like it" goes?

And what if the maintainer does not talk about any forum in the README? (What is, like, some 90% of the repos.)

<comment><USER19> Feature requests are fine, since, like bugs, they have a defined scope and will eventually be done. However, I've seen some major products move their feature requests to a platform like UserVoice that's designed to track votes and progress.

</comment></comment></comment></comment></comment><comment><USER21> "Once your revenue reaches a minimum of $500 USD for the month, you'll receive an electronic payment from GitHub for 75% of the sales price."

So it looks like Github takes a 25% fee. I thought maybe they would do something more innovative regarding the fee structure, or maybe have a lower intro fee like 15%. But here is seems pretty much on par with the 30% of the Heroku marketplace.

<comment><USER22> Wow, i think 25% is more than enough, they don't have costs that justify 25%. It's just their monopoly that allows them to charge that much.

<comment><USER23> Monopoly in what sense? There is certainly healthy competition in the hosted Git repository space.

<comment><USER24> You mean the ones that delete their databases without backups?

<comment><USER23> I assume you are referring to GitLab, so yes. GitLab and Bitbucket are both actively releasing new features and have sizable paid user bases. GitHub is the market leader, no question, but it not nearly a monopoly.

<comment><USER25> "Market" is an interesting concept here.

Do you compare all usage, or just their hosted versions (i.e. do on-premise installs count)?

Do you compare all usage, or just paid usage?

Can a company that's not even close to profitable be considered a "market leader"?

<comment><USER26> "Can a company that's not even close to profitable be considered a "market leader"?"

Sure it can. Uber and Twilio are generally considered, far and away, as market leaders in their respective spaces; Uber is massively unprofitable and Twilio was operating at a loss at the time of its IPO.

Market leadership is a function of production and capacity, not profitability.

</comment></comment></comment></comment></comment></comment></comment><comment><USER27> Nice to see the old "oops we didn't bother to check the if the 'subscribed' column was true when we imported the mailing list" trick isn't beneath GitHub either.

</comment><comment><USER28> This is really cool, and I just discovered sentry.io because of it which is something that we have wanted but hadn't yet put the time into finding.

<comment><USER29> Sentry is pretty awesome. There are a few similar services like Airbrake, Rollbar, etc.

</comment></comment><comment><USER30> "...fine-grained repository permissions with GitHub Apps...", excellent, this caused us more problems integrating than anything else. Especially for an organization, anything beyond 15-20 repos and they just couldn't give permission.

Edit: It only applies to Github Apps, we have an OAuth App. shame.

<comment><USER31> GitHub Apps also support authenticating users with OAuth, so you might want to check out if it makes sense to migrate.

[https://developer.github.com/apps/building- integrations/sett...](https://developer.github.com/apps/building- integrations/setting-up-a-new-integration/about-choosing-an-integration-type/) [https://developer.github.com/apps/building- integrations/sett...](https://developer.github.com/apps/building- integrations/setting-up-and-registering-github-apps/identifying-users-for- github-apps/)

</comment></comment><comment><USER16> REST v3 API supports conditional requests, that return 304 when content hasn't changed since last time you asked and don't count against rate limit. It means one can use a caching HTTP transport and benefit from that.

Is there any support for caching in the GraphQL v4 API? I'm not seeing anything about it in the docs.

<comment><USER32> If your query is complex and/or depends on sharded data, it can be difficult or impossible to cheaply (i.e., without re-executing the query) check whether the results have changed. That doesn't mean that the server couldn't execute the query, hash the results, and compare to an ETag from a client's If-None- Match: header, or some GraphQL equivalent of that (if it has it), thus saving network bandwidth. You just might not be able to save cycles and I/O on the server-side.

</comment></comment><comment><USER33> Github's Marketplace would be much better to Github's bottom line long term if they expanded the free plan to include private repos. I can see this working more to bitbucket's benefit if Atlassian gets more aggressive with its own marketplace offering.

</comment><comment><USER34> Anyone knows if they plan to deprecate V3 (which is likely) and when?

<comment><USER31> We don't have a timeline yet, but any updates will be posted on the API blog: [https://developer.github.com/changes/](https://developer.github.com/changes/)

<comment><USER18> Does that mean you do plan to sunset it?

<comment><USER35> Not trying to be rude, but why would someone build a V4 API and plan to keep the V3 API alongside it long term?

</comment></comment></comment></comment><comment><USER36> This feels like a Docker-esque move. Taking a product that people love, then thinking that they can conjure new features on top to grow more. The issue is, none of these new features really seems to be something that people want. So, it creates bloat and harms the core product.

</comment><comment><USER37> I guess that the main benefit of GitHub Marketplace would be the full-scale integration with GitHub Issue and Project board, launched last year. Consider a common situation: when CI fails, people need to reopen an issue manually, getting an error report via email. People who are in charge of project management may be seeking a better way to streamline the manual task, which probably increased since the introduction of Project board.

To be clear, I've been away from software development over the years, so I'm curious about the impact of Project board. Does it useful in the first place? Did it require manual tasks between other services such as CI? Do you, who is in charge of issue or board assignment, feel frustration using it?

</comment><comment><USER38> I wonder how this will work with GitHub Enterprise?

<comment><USER31> GraphQL will be available in GitHub Enterprise 2.10, and GitHub Apps will be available in 2.11. For Marketplace, we will be researching how to solve similar problems for Enterprise customers after this initial launch.

</comment></comment><comment><USER39> Can somebody from GitHub comment on this?

> Apps cannot actively persuade users away from GitHub.

This is listed as requirement from the "Requirements for listing an app on GitHub Marketplace" page. Is this saying your app should only support GitHub?

</comment><comment><USER40> The Listing requirements: [https://developer.github.com/apps/adding- integrations/listin...](https://developer.github.com/apps/adding- integrations/listing-apps-on-github-marketplace/requirements-for-listing-an- app-on-github-marketplace/) mention: \- OAuth Apps should have a minimum of 1000 users. \- GitHub Apps should have a minimum of 250 installations.

Does this mean you aren't listed on the Marketplace until you have 1000 users + 250 installations?

<comment><USER41> GitHub apps are slightly different than OAuth Apps.

GitHub Apps are per-repository "integrations" that don't perform actions on behalf of a specific user and are installed directly to a repo (with fine- grained permissions).

OAuth Apps are the classic "integrations" installed by a specific user and perform actions on behalf of that user.

I imagine they have lower requirements because they are new, more specialized, and likely to be installed a little less.

<comment><USER40> Thanks.

I also see that the security requirements are quite high. While its difficult to argue with source code security - here are some of the security requirements:

The standard annual risk assessment shall include, to the best of Developer's ability, the following:

(i) SOC 1 and/or SOC 2 audit report; (ii) 3rd party proof of PCI compliance (a certificate showing Developer's handling of credit card payments is compliant); (iii) Privacy Shield Attestation; (iv) ISO Certification or Cloud Security Alliance Self-Assessment; (v) Cloud Security Self Assessment; (vi) any information on subcontractor or vendor production datacenter(s), IaaS, PaaS, or private hosting providers, as required by GitHub based on data and services rendered; and (vii) Written responses and evidence of specific security requirements as outlined in this agreement

[https://help.github.com/articles/github-marketplace- develope...](https://help.github.com/articles/github-marketplace-developer- agreement/)

The GitHub Marketplace will be an exclusive place for a while with those requirements.

</comment></comment></comment><comment><USER42> Kinda random - anyone have experience scaling GraphQL APIs? It seems super difficult since you're basically giving clients the ability to arbitrarily query your data.

<comment><USER43> I never worked on Github scale but if your API is not open and you're in control of all of your clients. Take a look at Persisted Queries[1] . It enables you to whitelist all queries that your application uses.

[1]: [https://github.com/apollographql/persistgraphql](https://github.com/apollographql/persistgraphql)

</comment></comment><comment><USER44> If I install a free version and hit the limit does it auto upgrade me and charge me or am I just cut off from the service and told I should upgrade to keep using it?

<comment><USER45> I'm an owner at Codetree - one of the project management launch partners for Marketplace. We talked through this scenario with GH when we were integrating.

What happens today is the limits are enforced on the app side, not he GitHub side.

So for example our free plan is two users. If you wanted to add a third you'd need to go GH Marketplace and upgrade to the eg 10 user plan. That would trigger an upgrade in Codetree and your plan there would enable you to add up to ten users.

In short: you trigger upgrades and downgrades, not the tool vendor you're using or GH, and thresholds are stored in the tool vendors database so they're the ones who decide how to tell you that you need to upgrade.

</comment></comment><comment><USER46> How is GraphQL related to the marketplace?

<comment><USER47> It's not. Each section of the post is a separate announcement.

</comment></comment><comment><USER48> So this is a marketplace for "meta-apps" \- apps that run on codebases.

I am concerned if in the future Github will add private repos to the marketplace. Making this move will discourage people from sharing code freely.

</comment><comment><USER49> No CI? Gitlab and Bitbucket have built-in pipelines.

<comment><USER50> What's the point in GitHub creating yet-another-CI-tool? Travis and Circle CI have had good integrations for years and work very well.

<comment><USER49> Cost?

<comment><USER50> Circle CI works well on their free tier, so no, not cost.

<comment><USER49> If you're OK with one container, sure.

<comment><USER51> You can always purchase more plus open-source projects get four containers for free.

</comment></comment></comment></comment></comment></comment><comment><USER52> I did not followed recent GitHub news closely. Is GraphQL a new thing at GitHub? And do you think GitHub integration benefits from this?

<comment><USER53> GraphQL is a method of creating APIs for consumption by e.g. JavaScript apps or mobile apps that is gaining popularity at the moment. So it can be seen as a successor to REST. It gives you a pretty easy query language to specify which data on which objects you are interested in and that data is then fetched for you. An example could be a React app where each component specifies which data it needs to work on, and then when a page is loaded, all of the rendered component's queries are merged into a single query that fetches all the data in one go.

<comment><USER54> REST and GraphQL are two completely different things, and one is not a replacement for the other. REST is an architectural concept, whereas GraphQL is a query language spec.

[https://philsturgeon.uk/api/2017/01/24/graphql-vs-rest- overv...](https://philsturgeon.uk/api/2017/01/24/graphql-vs-rest-overview/)

<comment><USER55> That's true in theory, but in practice REST is a (limiting) spec for the kinds of verbs you're allowed to use between systems, and graphQL is a significant and much needed expansion of that.

<comment><USER56> > REST is a (limiting) spec

Nitpick but REST is not a spec to begin with. It's a set of ideas. There is absolutely no normative REST spec. Developers as of today are still debating what REST is, for better or worse. GraphQL IS a spec, like SOAP, XML-RPC ...

<comment><USER55> I agree that REST isn't a spec theoretically, but in my experience when you talk about building a Restful api you tend to be actually agreeing to use the HTTP verb spec, which is what I don't like at all.

</comment></comment></comment></comment></comment></comment><comment><USER57> I think it's only an upgrade for shitty REST-like protocol for APIs. The next after GraphQL will be something like executing your own code in sandboxed environment on the server, with strong constrains etc. Anyhow, I don't like centralized approaches. Easier access to Github data will encourage to create apps based on a global variable which is the Github. I'd rather want them to release installable libraries that communicate with other parts of the system. Sort of.

<comment><USER58> > The next after GraphQL will be something like executing your own code in > sandboxed environment on the server, with strong constrains etc.

Ah, like a SQL database with procedures? Honestly, I've wondered why the SQL interface isn't common for the remote APIs. My thoughts are that it's possible to run very expensive queries and you'd need provide some HTTP transport so it works for browser JavaScript.

<comment><USER57> I think we're don't do that because it's too dangerous and user would have too much privileges. I'm referring to something like AWS Lambda, but handled by your library on your web server. Say you can run JS snippet only for 10ms and the only global non-standards variables are those that contain data that can be queried. Or even pass them through function params, look:

ctx => ctx.users.filter(u=>u.name.contains('Janusz')).slice(0,10);

And that will return 10 firsts users as JSON.

</comment></comment></comment><comment><USER59> I expected GitHub to attempt further monetizing but perhaps not something as cheesy as an "app store".

</comment><comment><USER60> Better integration and monetization of Github Apps (formerly Integrations)?

</comment><comment><USER61> Is it all GPLv3'd? That's my question. If not, why not?

</comment><comment><USER18> The shameless support posts by execs/founders at launch partners are slimy! You guys should be ashamed. This isn't your marketing site, it's for discussion.

<comment><USER62> So what the hell is "Show HN" _but_ marketing?

<comment><USER25> Show HN is expected to be submitted by the person who made the thing to show, and it's expected that they're showing it for a reason: it's either new, or had a significant change.

Edit: So technically, GH probably _could_ have had their new shit posted as a Show HN: but you generally want that to be something people can 'play' with right away, not a blog post about a way to pay more for the same things. /Edit

My best guess with what happened here (based on the similarity of the texts) is that GH arranged a pre-formatted 'base' text, with placeholders, sent it to each of the involved CEOs/whatevers and said "we will submit our story to HN, once we do please comment using this template".

Every one of the comments basically matches this pseudo-regex: "<Name> (at|from) <Company> [1] ..... one of the ([:digit:] <App Category>)? launch partners" and the [1] is referenced as a link to GitHub Marketplace. Only one has a link to their own site.

The HN story itself was submitted by someone who at least used to be, and based on that, likely still is a GH employee.

I assume they were trying to drum up lots of positive attention about this with the cool-kid me-too cargo-culting developer set that makes up a large part of HN readers.

</comment></comment></comment><comment><USER25> The eerily similar posts from CEO/etc of "chosen partners" in this thread is a bit creepy.

"We are the chosen ones. We love the leader."

<comment><USER18> Beyond creepy. It's gross, shameless, and inappropriate content (marketing) for an HN comment.

</comment></comment><comment><USER63> Only for Mac or PC? Clearly not aimed at me

</comment><comment><USER64> So basically they just released GraphQL. Everything else is just renaming?

</comment><comment><USER65> There is some undesirable horizontal scrolling on this page for me (mbp 13") -- fixed by adding `overflow: hidden` to `div role="main"` element.

<comment><USER66> Just hiding bugs isn't a good practice! It'll ruin responsiveness with lower resolutions where that content is required.

Actually fix the underlying cause, not just add a quick hack to make it go away.</comment></comment>


Show HN: Free, open source JavaScript and mongo software to manage a Facebook page - <AUTHOR> http://blog.hookupjs.com/product/

<comment><AUTHOR> Nervous asking for feedback here; but, you can't learn if you don't ship and ask questions. So here goes, free, open source javascript & mongo DB, Angular, Node based system to score a Facebook page based on Insights data; schedule posts, see trends and create marketing, branded creative.

All free and functional today. Now, pelt me with the rotten fruit. I'm curious and begging for feedback. The online account on our site does not work, as it's not approved by Facebook. The code is functional, and it's what we've been installing on 2 macbooks and a Linux box; mostly Ubuntu and hosted by Digital Ocean.</comment>


Going old school: how I replaced Facebook with email - <AUTHOR> https://blog.chaddickerson.com/2019/01/09/replacing-facebook/ <comment><USER2> There's one aspect of Facebook which seems to always be ignored by these "deleting Facebook" posts and it confuses me.

I've never really scrolled through the frontpage, I don't look at people's posts and photos, happy to message them through text or WhatsApp, but I absolutely need the events. Without them, I'd lose so much touch with all but my closest friends.

People tend to respond to this concern with "if they don't invite you individually they aren't that great a friend" but that's my point - they're not, but they're people I like and I like to see them a few times a year at birthdays and other celebrations, and I'd feel my life diminished if I didn't have them. But there are dozens, I don't have time to catch up with them for coffee, and I don't honestly love the pressure of one-on-one time with people who aren't very close to me, anyway.

People who advocate deleting Facebook, do you not have/want such relationships? Or do you not care for catching up in that way? Or do you have some solution? Because for me it's the absolute core feature of Facebook.

<comment><USER3> I have been on Facebook for a decade and can count on one hand the number of events I cared about that I was invited to on FB. Maybe this is because I'm "old" (late 30s). Or maybe some friend groups simply do not use FB in this way. Most events I'm invited to are via email, evite, or text.

<comment><USER2> That makes sense, I wonder if it is just a generational thing (late 20s). While I'm aware of evites, I've never received one, and my circle of friends seem to basically set up a Facebook event and then forget about it.

I don't think I've ever received an invite to a casual event over text/email/phone either, except in conversations of the "Hey, what are you up to?"/"Going to a party, want to come?" ilk.

<comment><USER4> I am also younger (mid 20s) and I've had my FB deactivated for several years now; I also don't have Instagram or Snapchat. I admittedly do use Meetup.com and its app. Initially, I deactivated FB because I felt it was an addictive time-sink (as Semaphor mentioned elsewhere). Now, I spend a lot of time wondering whether I should completely delete it, because I honestly don't feel like I'm missing much.

I just text everyone I want to talk to.

Maybe it's because I went to a decently sized high school (a little under 2000). Or maybe it's because I currently live in a large city (top 20 populated city in the U.S.) Or maybe it's because I'm an extrovert.

But I would probably be overwhelmed by the sheer number of events my social network would produce on FB. I already accompany friends to events thrown by friends of friends pretty frequently.

How often do you go to these birthdays of people you barely hang out with? The average person has like 300 friends on FB. The median number is about 150-200. That's a lot of birthdays.

<comment><USER5> Friends on facebook doesnt mean friends in real life. Most peoples circle is less than 50.

</comment></comment></comment></comment></comment><comment><USER3> > _My email list is broadcast-only but any replies go directly to me._

This sort of mass-email is weirdly impersonal because it tells the recipient that you want them to know what you're up to, but you don't want to write a message just to them. Also, it would be very awkward to opt-out ("hey dude, um please don't send me your mass-email updates anymore."), unlike company emails which there's no awkwardness around unsubscribing.

I'm not saying this isn't better than FB, but it still has some downsides (some of which the author may be overlooking because they don't bother him).

<comment><USER6> > weirdly impersonal because it tells the recipient that you want them to know > what you're up to, but you don't want to write a message just to them. Also, > it would be very awkward to opt-out ("hey dude, um please don't send me your > mass-email updates anymore.")

You mean, like a post on FB ? Or Tweeting ? Readable for all 'friends' or 'followers' ? Who very likely are not really interested in your message ?

<comment><USER3> If you follow someone on Twitter, that means you want to get their updates. And it's trivially easy to mute people on Twitter/FB so that you can stop seeing people's messages without risking offending them or hurting their feelings.

Another reply mentions having people who don't want to read your email newsletter set up filters to get rid of them. This is possible, but more work than a two-click mute on social media platforms.

Don't get me wrong — I'm not saying FB/Twitter are "better" — just that this solution occupies a middle ground that creates different privacy/relationship issues.

<comment><USER7> > This is possible, but more work than a two-click mute on social media > platforms.

It was four clicks for me to set up an email filter on Gmail in response to a message in my inbox.

And there's considerably more filtering options, had I wanted something non- trivial, than there are on FB or Twitter, which are all or nothing -- after all, on email I have the option to more than mute, such as to classify the message based on various parameters.

</comment></comment></comment></comment><comment><USER8> I've deleted my Facebook account a few months ago after almost a decade of using it exclusively to sign up for random things. Clicking on the big blue button seemed easier than entering my e-mail and selecting a password, so I kept doing it until the privacy implications of this practice convinced me to abandon it.

However, I won't be starting a mailing list any time soon.

Perhaps others lead vastly more interesting lives than I do, but I don't feel any need to launch a personal mailing list about my life any more than I felt the need to share a stream of minutiae on Facebook. Moments that are genuinely dear to me - such as those relating to my family life - are none of anyone else's business anyway.

3,000+ contacts over 20 years? My goodness, that's a lot of people. I'm an immigrant 2x removed from his home country, from where I left too early in life to have developed relationships that could continue in the online age. I came to my current country too late to have gone to school here, so I don't have the implied social network which usually comes with that particular experience.

99% of people I've interacted with happen to have been colleagues of mine at one time or another, or folks I've met while sitting on either side of the job interview table. While I have enjoyed some very productive work arrangements in the past, none of these people have kept in touch with me after myself or them have moved on to another company.

All this is to say: I would be astonished if anyone wanted to read about the details of my life. These aren't useful - or indeed enjoyable - to anyone but me and my family, which is where the details shall remain.

</comment><comment><USER9> Forget going email - I am thinking of going back to snail mail. I received a written card invitation in the mail yesterday from a good friend [0] inviting our family to a dinner, and it felt indescribably nice to get it and read it.

Something nice about holding it in your hands and reading it, then placing it on my desk somewhere so I could see it and be reminded of the good feelings every time I glanced at it.

[0] - [https://twitter.com/dsabar/status/1085727043956830209](https://twitter.com/dsabar/status/1085727043956830209)

<comment><USER10> I write letters to a friend in the U.S. from NZ at least once every 2 months. If you don't have stamps and envelopes handy it can be annoying at times but there's something about spending an hour reading, re-reading and then writing a personal letter by hand.

Also, once you write something, there's no backspace. You either re-write the page completely or leave a mess.

</comment></comment><comment><USER11> For any social network to work you'd need most of your connections to get onboard the same thing, whether it's Facebook or anything else. It's easy for some people to start emailing their friends but it's hard to imagine that the general public start doing the same thing, and without that, the alternative won't go very far.

In fact I'd find it much more annoying than Facebook if I start receiving such emails from friends.

<comment><USER12> Here's the fundamental thing. We don't need digital social networks. Email, just like mail, is sufficent.

<comment><USER13> Pretty much. There is simply no reason why I need to know what all my friends are doing and how they are feeling every single day. I just wish we had a good alternative for event organizing because that imo is the greatest value in facebook.

<comment><USER14> _> I just wish we had a good alternative for event organizing_

There are many alternatives, see: [https://blog.thefetch.com/2013/05/28/10-cool-sites-for- event...](https://blog.thefetch.com/2013/05/28/10-cool-sites-for-event- organization/)

<comment><USER15> Sadly, the very fact that there are many alternatives means that none of them are good.

The goodness of an event planning platform comes from everyone using it. Metcalfe's law in action.

I say this as a non-Facebooker, fyi. Events kept me on the platform well after the rest of it became net-negative.

I do maintain my social graph and as a result use Messenger sometimes. If someone really wants to talk I go elsewhere for the conversation.

<comment><USER16> Does not necessarily means they're not good though. If those several alternatives are federated through activitypub and part of the fediverse, as Mobilizon will be when released, then strength is in the number of alternatives.

Then again Metcalfe's law has a flipside relative to the cost of exclusion from the network to those included and society, and this costs grows bigger as the dominant network grows bigger. Among the effects of such exclusion is the existence of parallel networks such as the alternatives mentioned here, and what makes them very good is that they cater to some of those excluded from the dominant network. Dark side of Metcalfe's law in action.

Had facebook not been an antisocial closed network preying on its users but open and federated as the internet has been deisgned then things would have been different, right now the problem seems to be that facebook captured a majority of internet users in some geographical areas and does everything possible to keep them captive.

</comment></comment></comment></comment></comment></comment><comment><USER17> For me, it's more like, "How I never replaced email with Facebook."

<comment><USER16> Same here. Though I had not foreseen that email would be replace by mobile number among the younger generations who often now do not even have email or do not use it or do not expect email to be used for actual people to people communication.

</comment></comment><comment><USER18> \- there are innate structures in relationship, and changing that (facebook blurs the naturally local/private lines) harms people

\- no beginning or end in the streaming era of web (unlike the slow static documents early model)

\- not knowing is ok, great even

facebook is an amplifier of huamn relationship, but amplitude is not a measure of quality

</comment><comment><USER19> Here's how I replaced Twitter with email.

[https://www.tinyfollow.com](https://www.tinyfollow.com)

e.g. Weekly digest of the tweets by Paul Graham: [https://i.imgur.com/kd8SCKv.png](https://i.imgur.com/kd8SCKv.png)

<comment><USER20> Wow, this is something I did not know I wanted. After having left Twitter a while I find myself missing some of the tweets from the more interesting people I followed.

I mainly stopped using Twitter because of the ads, slowly ruining third party clients (Tweetbot was what I used), and the tracking.

Thanks for sharing this!

<comment><USER16> Why not use rss-bridge[1] to make a rss feed for you twitter needs ? There are several public instances that allow you to do exactly that.

[1]:[https://github.com/RSS-Bridge/rss-bridge](https://github.com/RSS- Bridge/rss-bridge)

<comment><USER21> This is something I definitely wanted and didn't know about. Thanks!

</comment></comment></comment></comment><comment><USER22> I'm not sure what Mailchimp is doing in this scenario. Couldn't you just…you know…send an email? Using BCC, of course.

<comment><USER23> * Mail email hosts are not cool with sending messages to large quantities of recipients because it looks like spamming, sending via a host that exists for sending mass email solves that.

* The Mailchimp platform provided some automation for the author when first asking people to opt-in. Presumably each message also includes an Unsubscribe link at the bottom which is less awkward to use than replying to the author, asking that they stop sending the email.

* TFA mentions Mailchimp's templates, it's not easy to create readable formatting of lengthy emails in standard email clients.

* It eliminates the possibility of screwing up and addressing the email to recipients using a line other than BCC.

<comment><USER13> Mailchimp basically is a spam company, except its "legitimate spam" because you signed up for a website once and forgot to opt out of spam letters. I could block every single email from mailchimp and I doubt I would miss anything that wasn't spam.

</comment></comment></comment><comment><USER24> For me, Facebook is still the only way to keep up with my closest friends and family. I live 3k+ miles away from were I grew up, and it's Facebook's asynchronous form of communication that allows me to spend minimal effort to keep abreast of what's going on there.

Another thing that Facebook offers is the ability to get to know who I'm dealing with when I friend a new colleague or someone I just started hanging out with. It's a great way to spark conversation "oh hey, I see you made cocktails last night, I love run rummers!". Or, "I see you liked Datsik, want to go to a concert with me?". That's essentially how I met my wife.

</comment><comment><USER25> My family does monthly email updates mainly because not everyone is on Facebook. No noise. Just what's new with everyone, once in a while. There's no ads, no posts about what they had for lunch, no reposts about their political opinions, etc.

We don't even use mailing lists. We usually just reply to the last one, and I have a contact group with everyone in the family.

<comment><USER26> Who is "everyone" in your family? According to ancestry.com that's an absurdly large group.

<comment><USER25> Me, siblings, parents, grandparents, aunt, uncle, and a cousin.

</comment></comment></comment><comment><USER27> I made a chrome extension that blocks my Facebook news feed but still allows me to use messenger and see notifications. I'd estimate that I use FB ~90% less now. News feed was the biggest distraction — their algorithm is incredibly optimized to suck you into the rabbit hole.

My chrome extension in case anyone is interested: [https://chrome.google.com/webstore/detail/marathon/nkhecjgkf...](https://chrome.google.com/webstore/detail/marathon/nkhecjgkfpkkcejhbmfjghcokmhbmoed?authuser=1)

<comment><USER28> I did the same just by using a couple of element targets from withing uBlock origin.

Here they are for anyone interested:

www.facebook.com###u_jsonp_3_0 > div www.facebook.com###feed_stream_5a55a64611d#####0189168 > div www.facebook.com###stream_pagelet www.facebook.com###notificationsCountValue

The second one, you will likely have to figure out yourself - it looks like each feed stream has a unique id.

</comment></comment><comment><USER29> I've got a better idea. Write people you actually give a shit about an actual email or ... god forbid a letter or holiday card instead of mass broadcasting what you had for lunch today.

Worked for me!

</comment><comment><USER30> I once took a picture, and offered to email it to my younger co-worker. (I'm old.) He paused, and said "Email? I don't remember the last time I used email. Why don't you text it to me?"

There are people who just don't use email, and not because they're not tech savvy.

<comment><USER31> Given that it's required to sign up for online billing and most online services, I think your younger coworker either doesn't understand how to set it up on his/her phone or was just straight up lying because the email address was embarrassing or some other reason to hide it.

<comment><USER30> So you set it up, sign up for whatever, and never use it again except to sign up for something.

</comment></comment></comment><comment><USER32> Not sure how best to express this thought but it seems to my geriatric mind that some kind of p2p-ified (server+client in one) variation on NNTP could provide a "social network" with similar qualities to email but (unlike email), is something I pull from rather than gets pushed to me. Discoverability might be an issue though.

Is any of this making any sense?

<comment><USER33> Well, there's Scuttlebutt: [http://scuttlebutt.nz](http://scuttlebutt.nz)

Previous HN Discussion: [https://news.ycombinator.com/item?id=16877603](https://news.ycombinator.com/item?id=16877603)

</comment></comment><comment><USER34> Why stop there? [https://www-cs- faculty.stanford.edu/~knuth/email.html](https://www-cs- faculty.stanford.edu/~knuth/email.html)

<comment><USER35> It's nice to be in such a privileged position. Unfortunately not an option for most of us.

<comment><USER36> A secretary certainly sounds expensive. Even if you had a remote Secretary as a Service who prints directly to your home printer as an implicit inbox (which you can't hear printing, because that would be too much like getting a notification), it would still cost too much.

</comment></comment></comment><comment><USER37> > My email list is broadcast-only but any replies go directly to me.

If I were to do this, I actually think it would be cool if those in the mailing list could choose whether they want to reply to the whole list or to just me (as easy as removing the list from the CC line). That way, there can be a semi-public discourse with your friends, if you still want something like the "comment thread" from facebook.

<comment><USER26> Have you ever worked at a company with reply-all spam?

</comment></comment><comment><USER38> You can accomplish with something like TinyLetter ([https://tinyletter.com/](https://tinyletter.com/)) or Revue ([https://www.getrevue.co](https://www.getrevue.co)). Not sure why he went the slightly more difficult route of using a service like MailChimp. (which also happens to own TinyLetter)

</comment><comment><USER30> I send an email every few days to a small handful of friends and family. They're all on Facebook, I'm not. I put them all in the To: list. Some of them reply all, some only to me. It's all good. No software was written, no permission was asked.

This doesn't replace Facebook. It doesn't have to. Facebook is nuclear overkill for this little part of my life.

</comment><comment><USER39> Being in the same situation as the author (deleted FB account few months ago), I was just thinking how it would be nice to have a mailing list with friends&family in it but didn't know where to start, when this article hit top 30.

I would appreciate if someone would list few Mailchimp's pros and cons and/or maybe few alternatives, from privacy/freedom perspective.

<comment><USER40> Here's their privacy policy: [https://mailchimp.com/legal/privacy/](https://mailchimp.com/legal/privacy/)

Their service is good, but as far as I can see, they're willing to sell just about everybody's data to just about anybody.

</comment></comment><comment><USER41> I run a forum for my family. Don't have to worry about shit. People can use it as much as they want. Invite their friends, etc.

</comment><comment><USER42> Many of the downsides to this from the comments I've seen so far seem to be fixable with a new UI/UX for email clients.

Like if thunderbird had a "my lists" view - which put extended family in one area, close friends in another, work associates in another.. and the viewing of messages inlined pics nicely and did an infinite scroll kind of thing, basically creating a timeline/feed...

If someone could get ssl and pgp or similar in the mix, this could stick. Just maybe.

Actually I am sure there are lots of family groups that have things to send to each other that they don't want thousands of other people to access, and friends groups as well - so this could be federated snapchat kind of fb replacement with some UI tweaks, an app and a web portal.

NOw I wonder if email, can be secured and connected to mastadon, riot.im and similar to expand the options for viewing, as well as replying.

</comment><comment><USER43> I make photo-albums in plain HTML, use basic HTTP authentication, then send out the links via e-mail.

</comment><comment><USER44> He has an interesting idea in using a free MailChimp account. I would have thought of just having a local email list but it seems very 'tidy' to use MailChimp instead. I have set up MailChimp for a customer but never used it for myself.

I just put on my TODO list to start 2 lists: one for family and friends and one for interesting tech.

I will never totally leave FB because I use it to pimp my books and open source projects, but I think spending more that 10 minutes a week engaging with FB is a waste of time. Opportunity Cost.

</comment><comment><USER45> My family (parents, sister, some cousins) "replaced" Facebook with iCloud. Shared photo albums with comments and iMessage covers pretty much everything.

<comment><USER16> This bears some same issue facebook has:

1: Some of the people who matters may not have an apple device/account now or may not have one in the future

4: With Apple services, you're not free to switch platforms and your data is not on your own computer

</comment></comment><comment><USER46> I've been banging this drum for a while, but to me it's clear as day: the company that cracks this and makes the process described within easier and monetizable will be the next unicorn.

People are starving for ownership of a one-to-many communication address (which email is designed for, but has not been well executed by email clients since email has been seen first as business/office tech).

</comment><comment><USER47> Lesson 2: Email is more intimate and leads to better conversations.

Lesson 3: You control the narrative completely in email which provides a much better opportunity for story-telling.

Facebook Messenger?

Facebook isn't just a place to post publicly on your wall. Arguably the reason why Facebook messenger exists on the platform is provide intimacy.

<comment><USER16> Anything that exists on facebook is there to profit facebook not the user.

if facebook messenger exists, it is not to provide intimacy but for some reasons aligned with facebook interests such as getting the people online on facebook longer to peek further in their life and social links to feed the investor storytime narrative that facebook will collect more data to eventually be able to target people efficiently enough at some point in the future so that investor will make a big ROI.

Then again email vs instant messaging is an old debate and email does provide much better conversation and intimacy due to its asynchronous nature.

<comment><USER47> That's not what I or the user was referring to.

The user and I are referring to the difference in posting and discussing as a public post rather than talking through Messenger or email which is a more intimate service because its not broadcast publicly.

Not the information and data privacy behind what Facebook might do or not do with that information behind the scenes.

</comment></comment></comment><comment><USER48> why would you even share photos through horrible Facebook compression? i shared them with my family through Google photos before abandoning Google services, later sharing through WhatsApp (yes i know, horrible compression) and going to finally move to Signal since they started now to support sharing of photos/videos (though only in beta), until now you could share only one photo or video making it useless

also the people i wanted to stay in touch are anyway on WhatsApp, signal or at worst can email them, so i don't see any added value in Facebook

</comment><comment><USER49> I could care less about Facebook, but it's the only way (easy way) for me to stay in touch with old squad mates, and international friends.

<comment><USER50> You could? How much less could you care about Facebook?

</comment></comment><comment><USER51> My wife's family keep a group message chat going indefinitely; it seems to work for them.

<comment><USER26> That only works if everyone in the group likes each other well enough, and the boundary of the group is agreed by consensus -- not most families :-)

</comment></comment><comment><USER52> 社交媒体带来的碎片化阅读真的已经影响到了我们的生活,我现在开始有意识的避开即时社交平台,重要的事情会通过邮件解决

<comment><USER53> In case anybody's wondering, Google Translate says:

The fragmented reading brought by social media has really affected our lives. I am now consciously avoiding the instant social platform, and important things will be solved by mail.

</comment></comment><comment><USER54> I thought this was going to be a tech blog on how he hooked up his email address to Messenger

</comment><comment><USER55> I would use Google groups for this instead of a true newsletter product, but I totally agree with this approach. It's similar thinking to the philosophy behind ctolunches.com.

An earlier thread had me think about productizing this actually: [https://news.ycombinator.com/item?id=18585534](https://news.ycombinator.com/item?id=18585534)

My thought:

For a social network, I see it being a cross between a Google Group and a newsletter. * List my profile somewhere with a sign up box to "subscribe to updates and conversation from Miles and his network." * Every 3-6 weeks, I receive an email from the platform asking me what's been up with me, and I simply reply, which then gets sent to people who subscribed from updates from me. * Calendar integration -- when I add an event to my calendar, I have the option of including it in my next update to my email network to let them know I'm doing it. (ex: I'm speaking next month at some conference) * Community participation -- if anyone who subscribes to my email updates/network wants to email my network (ex: do you know anyone who...), I get an email saying my subscriber Bob wants to ask my network a connection, would you like to include it in your next email update to your network? If I respond yes, then it is also listed in my next email update to my network. Something like that would be awesome.

<comment><USER56> We built some of this functionality for FWD:Everyone, but haven't yet gotten it to a point where it's polished enough to release. One of the issues we ran into is that Gmail translates unicode characters into images, which are then difficult to resize to get them to properly match the surrounding typography.

We also didn't build this as a fully automated service, but rather as something that generates MJML that you then have to compile and embed within an HTML email. But it does let you embed all the replies to an email in a new outgoing email, which you can conceptually see here in this marketing email we sent out:

[https://www.alexkrupp.com/fwdeveryone_email_blast.html](https://www.alexkrupp.com/fwdeveryone_email_blast.html)</comment></comment>


34% of IT workers have gained more than 10 lbs. in their current positions - <AUTHOR> http://blog.channelinsider.com/content001/channel_chatter/it_workers_weigh_in_on_health_habits.html

<comment><USER2> I've actually started losing weight now. The trick was getting keyboard/monitor stand that can be adjusting to the standing position. It's not that we have to stay in one place, it's the sitting: <http://abcnews.go.com/Technology/DyeHard/story?id=3922069>

</comment><comment><USER3> There are lots of other jobs that involve minimal physical activity -- I think a major factor is that some IT workers have pretty poor diets and don't get much physical activity outside of work. Obviously that's a stereotype to some degree, but I think there's at least some truth to it.

</comment><comment><USER4> Maybe they should shift to a different position. Several times a day, even.

</comment><comment><USER5> On average, a man will tend to gain about 1 lb per year as they age. I wonder if 34% of IT workers have been in their current position for around 10 years?

<comment><USER6> Or if 33% remained the same weight. And the other 33% lost 10lbs.</comment></comment>


AWS Announces DynamoDB - <AUTHOR> http://aws.amazon.com/about-aws/whats-new/2012/01/18/aws-announces-dynamodb/ <comment><USER2> Better link: [http://www.allthingsdistributed.com/2012/01/amazon- dynamodb....](http://www.allthingsdistributed.com/2012/01/amazon- dynamodb.html)

This actually explains the difference between SimpleDB and DynamoDB. Interestingly, (among other things) it seems that developers complained about the need to account for eventual consistency in their application, so they're providing a consistent data store.

Doesn't that just sweep the latency tradeoff under the rug, or is flash making up the difference? What about the availability tradeoff? (I like the formulation here, consistency vs availability and consistency vs latency, as opposed to CAP which never made sense as a 3-way tradeoff: [http://dbmsmusings.blogspot.com/2010/04/problems-with-cap- an...](http://dbmsmusings.blogspot.com/2010/04/problems-with-cap-and-yahoos- little.html))</comment>


Interview with U. of Toronto professor and counterespionage hacker Ron Deibert - <AUTHOR> http://www.theglobeandmail.com/report-on-business/rob-magazine/why-canadas-hacker-king-is-very-afraid/article34471769/ <comment><USER2> So, this article is kind of a silly puff piece but the Citizen Lab has a strong track record.

It was the key part of a project called the OpenNet Initiative [1] which was in, in its day, the most comprehensive catalog of state-sponsored Internet censorship activities. It basically involved creating lists of sensitive web pages in various categories and then systemically trying to access them from within as many countries as possible, to see what was blocked and how it evolved over time.

They have done a lot of original research on what commercial tools are used by repressive governments to spy on or otherwise compromise the online activities of their citizens. [2]

They were instrumental in documenting and exposing GhostNet. [3]

They published one of the first comprehensive research reports on the extent of surveillance on TOM-Skype (basically "Chinese Skype").

They have done critically important research / investigative journalism on exposing the activities of CSEC, Canada's equivalent of the NSA, in spying on Canadians. [4]

Citizen Lab is like a combination of a security / threat research consultancy with an international relations thinktank that lives at a university. If this article seems like vague claptrap, it's because it's kind of difficult to describe their mission. They are a skeleton crew operation that is highly driven by the interests of the handful of researchers working there at any given time.

Disclaimer: I did a very small amount of totally insignificant work for them in 2010 as a undergraduate student at U of T.

[1] [https://en.wikipedia.org/wiki/OpenNet_Initiative](https://en.wikipedia.org/wiki/OpenNet_Initiative) [2] [http://www.cbc.ca/news/technology/netsweeper-bahrain- governm...](http://www.cbc.ca/news/technology/netsweeper-bahrain-government- internet-censorship-1.3769803) [3] [https://en.wikipedia.org/wiki/GhostNet](https://en.wikipedia.org/wiki/GhostNet) [4] [http://news.nationalpost.com/full-comment/christopher- parson...](http://news.nationalpost.com/full-comment/christopher-parsons- canada-has-a-spy-problem)

</comment><comment><USER3> The word hacker needs to be adopted in broader context. Hacking is essentially a creative process and not always a malicious activity. I would suggest software developers and makers to change their titles to 'hackers' of various types.

<comment><USER4> "Maker" is a good alternative to hacker though. It describes what people are doing. I suspect software developers want to use "hacker" instead because they actually _like_ the illicit "doing some bad and wrong and so damn cool" connotations that the media ascribes the term.

<comment><USER5> But I really don't make anything. More often than not, I'm taking things apart, looking at the guts, and trying to figure out how to use something. Sometimes those things break, and I have to go buy a new one.

</comment></comment></comment><comment><USER6> The first picture and the GIF made a quick scroll of this seem very unappealing. No idea who this guy is, but this piece seems like a Hollywood marketing bit.

<comment><USER7> That was also my first impression.

The main conclusion I drew is that even if it's an article about hackers, it's not written for hackers. It's written for the people who believe great hacking comes from geniuses pensively looking out their windows while dodging shadowy figures in suits.</comment></comment>