Web Scraping a Javascript Heavy Website: Keeping Things Simple - <AUTHOR> http://tubes.io/blog/2013/08/28/web-scraping-javascript-heavy-website-keeping-things-simple/ <comment><USER2> I used to use the network tab for stuff like this, but now I almost exclusively use mitmproxy[0]. Once things get sufficiently complicated, the constant scrolling and clicking around in the network tab feels tedious. Plus it's difficult to capture activity if a site has popups or multiple windows. mitmproxy solves these problems and also has a ton more features like replaying requests and saving to files. My ideal tool involves something that translates mitmdump into code that performs the equivalent raw HTTP requests (e.g. using python's requests). Sort of like Selenium's IDE but for super lightweight scraping.

[0] [http://mitmproxy.org/](http://mitmproxy.org/)

<comment><USER3> mitmproxy sounds like a lot of overhead if all you want is your own raw HTTP traffic. You can get this without Python, and without mitmproxy. Also, I thought mitmproxy was intended for HTTPS. Even in that case, I'm not sure installing Python and mitmproxy is necessary if all you want is to view your own traffic. You can just run your own CA and a proxy that can terminate SSL (e.g., haproxy).

Below is a simple, _lightweight_ ngrep solution. RE means a regular expression. This only saves packets with the RE you specify and does not save full packets, only the HTTP headers. 1024 is an arbitrary size to get all HTTP headers; adjust to taste. tcpdump is there only because ngrep does not work well with PPPoE. If you don't use PPPoE you don't need to include tcpdump.

         case $# in
         1)
         # capture HTTP headers to pcap file
         tcpdump -Ulvvvns1024 -w- tcp 2>/dev/null \
         |ngrep -O$1 -qtWbyline 'GET|POST|HEAD' >/dev/null
         ;;
         2)
         # search HTTP headers in pcap file
         ngrep -Wbyline -qtI$1 $2
         ;;
         *)
         echo usage: $0 pcap-file \[RE\]
         esac

To dump your results, try

         $0 pcap-file . |less

And here's a little script to make URL's from your pcap file. unvis just decodes URL's from the specs in RFC's 1808 and 1866. It assumes [http://](http://) URL's (no ftp://). The awk script ensures all URL's (not just consecutive ones) are unique.

        case $# in
        [12])
        above-script $1 ${2-.} \
        |sed -n '
        /GET/p;
        /Host: /p;
        '  \
        |tr '\012' '\040' \
        |sed 's/GET/\
        &/g' \
        |awk '
        !($0 in a){a[$0];print "http://"$5$2}
        ' \
        |sed '
        s/%25/%/g;
        s/\.\//\//;
        ' \
        |unvis -hH \
        |sed '/^http:[/][/]./!d;
        s/ /%20/g' \
        ;;
        *)
        echo usage: $0 pcap-file \[RE\] >&2
        esac

It's trivial to dump HTTP. You can feed this to netcat (using sed to modify the HTML to your liking), then open the result in your browser. Whatever you are aiming to do (I'm still not exactly sure - can you give an example?), I reckon it can be automated without Python and heaps of libraries.

<comment><USER4> Wow I thought mitmproxy looked rough until I saw tcpdump/ngrep/awk? They both work of course but neither look especially easy to use.

We've been using [http://www.charlesproxy.com/](http://www.charlesproxy.com/) for years, great tool (cheap albeit not free)

<comment><USER3> The parent asked for "lightweight". Have you considered the size of the charlesproxy binaries?

</comment></comment></comment></comment><comment><USER5> In many cases websites that load data asynchronously through an API are much nicer to scrape, as the data is already structured for you. You don't have to go through the pain of extracting data from a mess of tables, divs and spans.

</comment><comment><USER6> I've done a lot of scraping. Some sites use heavy Javascript frameworks that generate session IDs and request IDs that the XHR requests use to "authenticate" the request. In these situations, the amount of work to reverse engineer that workflow is pretty large. In these situations, I lean on headless Selenium. I know there are some lighter solutions, but Selenium offers some distinct advantages:

1) lot of library support, in multiple languages

2) without having to fake UAs, etc, the requests look more like a regular user (all media assets downloaded, normal browser UA, etc)

3) simple clustering: setting up a Selenium grid is very easy, and switching from local instance of Selenium to using the grid requires very little code change (1 line in most cases)

<comment><USER7> HtmlUnit† is also effective in such cases. HtmlUnit is intended to automate testing of websites. However, the very facilities that enable it to be useful for that purpose also make it useful for scraping.

A few years ago, I wanted to analyze retail store customer feedback data collected by a third party company. The stores were franchises, and the third party was anointed by the franchising company. The data was presented to the user (franchisee store management) via a fancy web site with its own opinion about how the data should be analyzed. My opinion differed. I wanted the data in low-level, RDBMS-friendly form, so that I could recast it every which way (and come back and do it again a new way I thought of). However, such was not forthcoming (big company, little franchisee).

The solution was to make a robot that put the third party company's portal through its paces at the finest granularity, scraping the numbers into a DB as they tediously appeared. The robot was in JRuby††, allowing access to HtmlUnit's functionality without the tedium of Java coding. It was slow, but I didn't care — run it overnight once a month, then run reports off the DB generated.

The coding approach was simple: Pretend you are a user. Access each page, starting with the login page, and do what the user would do. Scrape the interesting numbers as they appear. Append appropriate rows to the DB.

†[http://htmlunit.sourceforge.net/](http://htmlunit.sourceforge.net/)

††[http://jruby.org/](http://jruby.org/)

</comment></comment><comment><USER8> Before any naysayers complain about the idea of using undocumented endpoints, keep in mind that this is all in the context of web scraping.

</comment><comment><USER9> I've recently been learning all this the hard way.

1\. Documented API. Failing that...

2\. HTTP client fetching structured data (XHR calls). Failing that...

3\. HTTP client fetching and scraping HTML documents. Failing that...

4\. Headless browser

I recently found myself pushed to #4 to handle sites with over-complex JS or anti-automation techniques.

</comment><comment><USER10> If you liked this article, you might also be interested in "Scraping Web Sites which Dynamically Load Data" [http://blog.databigbang.com/scraping-web-sites- which-dynamic...](http://blog.databigbang.com/scraping-web-sites-which- dynamically-load-data/)

</comment><comment><USER11> For JS heavy sites, I've found proxying the traffic through Fiddler is the easiest way to discover the API end points I need to hit.

</comment><comment><USER12> I'm getting a 404 - Page not found

<comment><USER7> [http://tubes.io/blog/2013/08/29/web-scraping-javascript- heav...](http://tubes.io/blog/2013/08/29/web-scraping-javascript-heavy- website-keeping-things-simple/)

<comment><USER13> Still the link doesn't work

<comment><USER7> The original link is now working. He flipped the date in the URL from the 28th to the 29th, then back again.</comment></comment></comment></comment>


Apple Magic Trackpad - <AUTHOR> http://www.apple.com/magictrackpad/ <comment><USER4> Back in the days, I always preferred the Trackpoints the IBM ThinkPads used and I never quite liked the trackpads. I'm even going as far as to say that I'm quite the trackpad hater - they never felt precise to me and required way too frantic finger movement to be really comfortable for me.

But when I moved to Apple hardware, I basically was stuck with the trackpads, but this time around, it was different. The bigger size meant less finger movement and with the advent of multi touch, it suddenly felt like a more "natural" way to control the machine.

With the newer Macbook pros came this "the trackpad is the button" functionality and while it took me a day or so of getting used to, I absolutely cannot work with the dedicated button pads any more. Seing how quickly this became muscle memory, I assume that they did something (no idea what) right when they moved to that design.

This is how I went from "I hate trackpads" to "trackpads are the only really natural way to control the machine".

With this in mind, this new Apple device really interests me and I can't wait to get my hands on one. At worst it'll be a companion to my trackball, at best, it'll replace it (still also having a mouse connected for the eventual UT match though)

<comment><USER5> So, Apple trackpads are better in that they are more precise? Not being a Mac user, it's hard for me to understand what advantage a trackpad like this gives over any other third-party trackpad, other than the multitouch capability.

Please help me understand why people are gushing over a $70 trackpad (even made by Apple), when most people mock similar input devices costing over $30.

<comment><USER6> Two reasons.

First, external multitouch trackpads have been basically unobtainable until now. Apple bought the one company that used to make them and incorporated their technology into Apple products years ago. The only other model I've seen was an obscure Japanese import that only supported Windows. If you like trackpads, the Magic Trackpad is the first time in years you can buy one for your desktop.

Second, Apple makes the best trackpads in the industry by a huge margin. They're smooth and precise to a degree that PC laptop trackpads seem incapable of matching. The experience is comparable to the touchscreen on an iPhone or iPad. If you're not a Mac user it can be hard to appreciate the difference; I'm always shocked at how awful PC trackpads are every time I use non-Apple hardware. I'm used to hearing people complain about how much they hate trackpads, but if everyone had an Apple trackpad I suspect opinions would be quite different.

<comment><USER7> >Apple makes the best trackpads in the industry by a huge margin.

Do you have any research to back that up? I don't doubt that Apple trackpads are of high quality, probably even the best.

That said, the placebo effect can be very powerful when attempting to gauge how easy it is to interact with a computer.

<comment><USER8> I honestly don't think it's placebo. MacBooks are the only laptop I've ever owned where I've never felt inclined to use a mouse instead of the trackpad. They really are that good.

<comment><USER9> >I honestly don't think it's placebo.

Well if you did, it wouldn't work!

<comment><USER10> I've used a lot of pointers (keyboard, joystick, mouse, trackball, trackpad, multitouch, retinal motion, wacom) on a lot of operating systems (TI-94A, solid-state radar displays, Apple II, whatever was behind ACDS and CEC, Windows, Linux, Solaris, Mac, FreeBSD, etc, etc) in a lot of different contexts (tracking targets, directing fire, games, fetal ultrasound, planning and reading MRI, CT, Da Vinci machines, surgical microscopes) etc, etc, etc.

I don't think the Mac trackpad is anywhere close to a placebo. It's a parachute issue. (<http://www.bmj.com/cgi/content/abstract/327/7429/1459>). For the platform (personal computing), the mac trackpad is a hands-down winner.

<comment><USER8> Keep in mind though, it only really performs that well in OSX. In Windows there are gestures that clash with each other, in particular trying to right click a link in Chrome causes the window to scroll, very frustrating.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> I can't get my head around how Apple manages to come out with products that are cool enough they could come from a startup with some very motivated founders working day and night.

Almost all other companies with >100 employees are turning out mostly crappy products that no one gets excited about. When's the last time you talked about a Xerox printer or HP laptop? In my mind this is why a company like Nokia can't make a comeback into the smartphone market: they're probably unable to motivate a small team and then let them work out a cool product. Or they can't amortize away the risk of that team being on the wrong track, the way most startups are.

Apple can consistently pull this off, how do they do it?

<comment><USER12> Because people will buy their products on faith, where a startup would just be ignored. Belkin could release this exact same thing and it would be mocked.

<comment><USER6> Apple's multitouch technology comes from a startup called Fingerworks that Apple bought years back. Fingerworks trackpads were so well liked that they're still selling on eBay for hundreds of dollars.

The difference between Belkin and Apple is that Belkin has a history of mediocrity, while Apple does not. Apple earned their enormous reservoir of good will through years of great products.

<comment><USER12> Well fine, replace belkin with fingerworks. How many people have even heard of their trackpad?

Apple has the power to move something out of a niche market.

</comment></comment></comment></comment><comment><USER13> I actually think combining the Magic Mouse with Better Touch Tool (<http://blog.boastr.net/?page_id=1722>) is a better solution. I use the trackpad on my Macbook because it's more convenient than using a mouse when I'm on the go. In the office I prefer a mouse for precision, but I also want multi-touch gestures, and with Better Touch Tool I can pretty much customize my Magic Mouse any way I want to, which actually lends more credence to the hyperbole "Magic."

I have mine set up to move from tab to tab on two-finger swipes to the left and right, refresh the page with a three-fingered swipe up, close the tab with a three-finger swipe down, and two fingers up and down will activate expose and spaces respectively. There are a lot more gestures available to customize as well, I'm irritated Apple doesn't allow you that level of customization out of the box.

So with all that said, I think the track pad doesn't add much value, at least for me. My advice is to get the Magic Mouse, unless you actually hate it as a mouse (which some people do).

</comment><comment><USER14> Finally some of the stuff from fingerworks starts to surface as add-ons products for desktops. I've been wanting something to replace my long dead fingerworks keyboard, so I for one am happy.

Now... if only there were drivers for something other than just OSX.

<comment><USER15> I would replace my mouse at work with this in a heartbeat, but I'm a .net developer by day and _need_ those windows drivers. (I use an MBP at home)

I think they're missing a huge opportunity here by not providing those drivers.

Obviously an opportunity they have decided they don't care about, I just wish they did.

<comment><USER16> Maybe if enough people ask? Or the product is generally popular enough?

The iPod and the iPhone are only so popular because they embraced PC in all its ... glory. Hopefully they're already working on this.

<comment><USER17> The iPod didn't support Windows when it was released. Maybe Apple are doing the same thing here; I'm sure by not supporting Windows, they're able to get a better product out much quicker.

<comment><USER15> Fair point. It would be nice if they would at least commit to bringing Windows support in the future.

I can't imagine this would affect actual sales though, just another of example of when Apple's secrecy can be irritating for some of us.

<comment><USER18> For Apple to commit to anything in advance, let alone Windows support for a brand-new peripheral, would be evidence of takeover by pod people, and thus not nice at all.

Windows support for the original iPod arrived two years later.

<comment><USER19> Again, they were testing the market with the true believers. The music industry viewed the 2-3 million mac users back then as a nice microcosm they could experiment on. If it failed, it wouldn't get much press because back then nobody used macs.

</comment></comment></comment></comment></comment></comment></comment><comment><USER20> Looks like they're bringing out their version of the iGesture pad: [http://www.amazon.com/Fingerworks-IGESTURENUMPAD-iGesture- Pa...](http://www.amazon.com/Fingerworks-IGESTURENUMPAD-iGesture- Pad/dp/B00013MVT4). Apple acquired Fingerworks in 2005, which is how they got the multitouch technology used in iPhones, MacBooks, and now the Magic Trackpad.

The thing I'm wondering: My iGesture pad still works, and still has _way_ more gestures than the Trackpad. For example - twisting clockwise with four fingers is "close" (it presses command-w for you), which is pretty natural. Will the Magic Trackpad be able to handle this with software updates, or will we have to hack it to do the things its predecessor could do 5 years ago?

Also, be sure to check out all of the crazy multitouch products Fingerworks had before they were acquired. My favorite was the _insane_ keyboard at the bottom of <http://fingerworks.com/>.

<comment><USER21> I've owned <http://www.fingerworks.com/igesture_tech.html> for around 5 years. SOLID product. I'm glad I got one before they were acquired. It's definitely saved my mousing hand. And has way more features than Apple's product. Still, the iGesture cost me around $400 here in Canada. So to get something even close for $70, isn't bad.

</comment></comment><comment><USER22> I'm amazed that Apple seems so set on the AA battery format for desk peripherals that it is releasing a AA battery charger with the trackpad. I don't find AA objectionable, but Apple's marketing regarding removable batteries centers around "we can pack more battery in by eliminating the packaging."

<comment><USER23> My guess is that it has to do with the battery losing its ability to hold charge with time. It makes sense to engineer a massive new battery for the iPhone or Macbook, but these peripherals don't have the same volume or margin cushion to absorb the cost, hence off the shelf battery technology.

Also, I think most people expect to change out a computer every couple years, but a mouse is supposed to last longer. If the thing became incapable of holding charge after 18 months it would get customers riled.

After using a Magic Mouse for 6 months, I can say the rechargeable battery station is a must buy, I have to change my MM batteries every 3 weeks or so.

<comment><USER24> Having used both, I am disappointed with the longer lifespan of disposable batteries. I hate throwing them away, but with my usage they can last almost twice as long. On the other hand, I might mind less if I had a better charging configuration.

<comment><USER25> You can buy hight quality batteries that last long more than alcalines. I use varta +2100mAh.

</comment></comment></comment></comment><comment><USER26> As a lefty this would be awesome.

{Magic Trackpad}{Keyboard}{Mouse}

Ultimate productivity!

<comment><USER4> this is an awesome idea.

I'm able to use both hands equally well (though I might be hinging a tad bit to prefer the left hand). This could mean that I can alternate my pointing hand here and then

</comment></comment><comment><USER27> What I want just a bit more than this: an iPad app that emulates the Trackpad and does just a bit more.

For starters, a command-key could reconfigure it into a numeric keypad. Then incorporate custom per-app controls. (There are already iPad apps that do this.)

Such an iPad app would have the promise of the Optimus OLED keyboard. It would be much better, since high fabrication costs wouldn't be a barrier. It would have the potential to be a game-changer in terms of PC productivity.

Imagine integration with Eclipse for such a device. For example, when you execute your app in debug mode, it could configure itself as a debug remote. When you switch to an editor window, it could display editor macros and shortcuts.

Alternative: a 5" or 7" multitouch LCD device with a similar form factor to the Trackpad that doubles as a universal remote.

<comment><USER28> You don't have to look at a trackpad to use it.

I think that putting screens on trackpads, keyboards, or remotes (you use them to control _other screens_ ) is a horrible idea.

<comment><USER27> _You don't have to look at a trackpad to use it._

Looking is optional for the device I described. You'd only have to look in moments of managing greater complexity, which is the correct time.

 _I think that putting screens on trackpads, keyboards, or remotes (you use them to control other screens) is a horrible idea._

For a DVR, the second screen is very useful. You get context-driven visual feedback _without_ impinging on the video content in the main screen. Someone could check on a game or look up a related Wikipedia entry without interrupting everyone else's viewing. Someone could queue the next YouTube bit without the same.

</comment></comment></comment><comment><USER29> Is there _any_ chance of this pairing with a non-Apple desktop? I'd dearly love to join in the multitouch trackpad fun, but my main workstations are Ubuntu.

<comment><USER30> There's been some work on writing multitouch drivers for Linux: <http://lii- enac.fr/en/projects/shareit/linux.html>

It's a bit of effort because it requires new functionality throughout the software stack, from the low-level kernel drivers to higher-level X hooks, and ideally in the future, hooks in UI toolkits to do more application-level things with gestures.

</comment></comment><comment><USER31> Does anyone know if this will have some sort of way to trigger a "middle click", like when one clicks on the scroll wheel? As far as I've seen with Apple mice, they've never had support for middle click.

<comment><USER18> The Magic Mouse supports a three-finger click, which is interpreted as a middle click.

<comment><USER32> Uhh, no it doesn't. At least not without third party software.

<comment><USER18> Fair enough. The Magic Mouse hardware supports a three-finger click, and any of the various inexpensive third-party packages like jitouch et al make the Magic Mouse a viable input device.

</comment></comment></comment></comment><comment><USER33> Please Apple, gain the cut, copy, and paste (and other gestures) on the trackpad that you bought from FingerWorks. Freaking genius.

<comment><USER34> You can get 3rd party tools to configure this if you want. Use something like BetterTouchTool if you want to configure a gesture to fire up GMail. Right now I have a 3-finger swipe up for copy, and 3-finger swipe down for paste. I also have a 4-finger gesture for switching between tabs in google chrome.

However, I think there's probably a limit on how many gestures you can keep in your head, it's a lot easier remembering keyboard shortcuts than gestures. I can probably only keep track of five different gestures per app, but I'm sure I can probably recall at least 20 different keyboard combos when I use a tool like emacs.

</comment></comment><comment><USER35> I have some doubts that this will be comfortable to use for hours at a time. The Magic Mouse is bad on the wrists because of it's low profile. I have to wonder if the same will happen here. I suppose a bit of experience with using it is what it will take.

<comment><USER36> I use a bean bag under my wrist when I use the Wacom touch pad. Works great.

</comment></comment><comment><USER37> Hmm. Maybe for my Mac Mini in my entertainment center, but I much prefer a trackball at the desk.

No doubt BetterTouchTool will be updated to support this, making it far more useful: <http://blog.boastr.net/>

<comment><USER38> Yeah, I could see using it for my Mac Mini, I just wish you could somehow attach it to the wireless keyboard. Having two separate devices on your lap doesn't really work...

</comment></comment><comment><USER39> For wireless peripherals, non-charging AA batteries are a better choice than any charging dock or solution.

Why spend every day docking/undock the thing (or charging the batteries when they die) when I can slip new batteries in once every 6 months?

<comment><USER40> If you have a digital camera of some sort then you're probably already accustomed to charging batteries anyway. I think recharging batteries is less messy than buying new ones each time. You have to stock them ahead of time (storage area), and remember to restock after resupplying your device (which you'll probably forget). Don't forget that each time you have to access new batteries you'll need to fight with those plastic containers and chances are you won't have scissors on you, use knife instead and kill or wound yourself in the process.

Also, environment.

<comment><USER39> I'm inclined to agree with you regarding devices that you take outside the house, like a digital camera. My comment was more about desktop peripherals. To address your points:

1) A 24 pack of AA batteries somewhere near your desk should last over a year, and you should have storage area somewhere to accommodate it.

2) AA batteries generally come in easy-to-rip cardboard packs, so no fighting with the container and hurting yourself.

3) The environment argument could fall either way; I suspect neither of us has seen definitive research regarding the total environmental cost of disposable vs. rechargeable batteries.

<comment><USER40> For devices that you keep at home any kind of batteries are really just a workaround for actual wireless electricity. Anyway my point is that most people that buy wireless trackpads probably already own power-sucking devices like digital cameras and so have invested, and are accustomed, to charging their batteries. In this case keeping additional non-rechargeable batteries wouldn't provide any benefits and would only add to confusion, as you have to be careful not to mix them.

If you are not already enslaved to your digital camera's power inefficiency then perhaps buying brand new batteries might be more comfortable.

In regard to environment your 24 pack of AAs wouldn't probably have any significant difference whatsoever and you won't live long enough to observe any. The impression, though, that rechargeable batteries are more environmental friendly will provide you with nice dopamine boost each time your device needs resupplying.

</comment></comment></comment></comment><comment><USER22> I wonder:

* If it will be Inkwell compatible someday if not now: <http://www.apple.com/accessibility/macosx/physical.html>

* If it will BT pair with iPhone like the BT keyboard.

<comment><USER41> No, just like the existing trackpads on the MacBook Pro, it's not Inkwell compatible.

And No, since you can't pair a bluetooth mouse with the iPhone, you also won't be able to pair a trackpad.

</comment></comment><comment><USER42> next step is to make a one giant trackpad with keys printed on top of it. os if you touch it its a key press and if you drag it its a mouse movement. now only need to figure out how to mouse click ;)

<comment><USER43> Fingerworks had a keyboard exactly like that, many years ago:

<http://www.fingerworks.com/>

They got bought by Apple for their multi-touch experience.

</comment></comment><comment><USER44> I've been waiting years for this. YAY.

</comment><comment><USER45> I was wondering when Apple would get around to doing that. I want Apple's version of this - [http://www.google.com/products/catalog?&cid=102343875510...](http://www.google.com/products/catalog?&cid=10234387551099087518#) \- Keyboard + trackpoint/pad, and make an external usb keyboard out of it.

</comment><comment><USER46> I'd really like it if two of these would play nice with other. I could see doubling up with one on each side of my keyboard. (Been thinking about this for a while.) I know little to nothing about hardware (UX/marketing guy) does any one more knowledgeable consider this use of the driver probable as built in functionality?

</comment><comment><USER47> Does anybody know the dimensions of this trackpad? I've been looking for a way to accomplish something like the image below with 2-finger scrolling (either above or below the thumbkeys) on my Kinesis Advantage:

<http://www.flickr.com/photos/robert-p/4408732537/>

</comment><comment><USER48> I love this idea, but how come they couldn't include this functionality into the Magic Mouse too? (Maybe next version?)

</comment><comment><USER49> How are Apple trackpads in terms of ergonomics? When I first started using a laptop 5 years back, I found myself feeling massive pain in my wrists as a result of using the touchpad. To this day, I find a mouse far more comfortable to use; however, I haven't had much of a chance to try out the touchpad on Macs.

</comment><comment><USER50> Engadget review:

[http://www.engadget.com/2010/07/27/apple-magic-trackpad- firs...](http://www.engadget.com/2010/07/27/apple-magic-trackpad-first-hands- on/)

"It feels just like using a slightly larger version of a MacBook or MacBook Pro pad. And we mean exactly, right down to the multitouch gestures and whole- pad click."

</comment><comment><USER51> Now that all macs can take advantage of gestures, I wonder when Apple will include some of the iOS inertial scrolling to OS X. It's such a simple change that (for me) makes a big difference in functionality. I was happy to see inertial scrolling added to Google Maps not too long ago.

<comment><USER52> The magic mouse has that type of scrolling as a setting you can enable.

</comment></comment><comment><USER53> I've been waiting for this since the rumors first started surfacing. As someone who uses a $100+ Kensington trackball, this is a bargain.

If anyone has information on Windows 7 drivers for the Magic Mouse (whichare presumably the same), let us know!

Otherwise I'll have to write them myself. :)

</comment><comment><USER54> Can it track eleven points of contact like MacBooks?

[http://www.gottabemobile.com/2009/11/09/track-11-fingers- on-...](http://www.gottabemobile.com/2009/11/09/track-11-fingers-on-your- macbook-inconceivable/)

<comment><USER55> They did say it was the exact same touchpad, just bigger... so very likely.

</comment></comment><comment><USER56> If you use BetterTouchTool or MagicPrefs with your Magic Mouse you can get all those touch features in your mouse, that is definitely better when it comes to move a cursor.

<comment><USER57> I think it's a matter of preference. I've been saying how much I wanted to ditch my mouse for a desktop version of the MBP trackpad since I got my laptop.

<comment><USER15> Here Here. I used to hate trackpads, so much so that I only ever used laptops with "nipples" (see ThinkPad).

The MacBook with two-finger scroll was revolutionary for me, and I've never seen any manufacturer beat that pad.

Well, any _other_ manufacturer. The glass trackpad on my MBP is something else, and this is just a bigger version of that. Awesome.

Apple are unbelievably ahead of the curve on this, I'm not quite sure why the other manufacturers are failing so badly. The trackpad on my HP workstations are as-good-as unusable in comparison. I always have to use a mouse.

</comment></comment></comment><comment><USER58> I dont know, it just seems weird to be positioned on the right (where the mouse is)... well, won't know till i try it~

<comment><USER55> Just because nobody's really done it before... it does seem weird - <USER2> if you want to use a mouse.

\- It's cordless - you can put it on the left or right or anywhere else. \- You don't strictly need a mouse.

\- I remember thinking the touch-surface on the magic moues was silly -but now I miss it when Idon't have it... and I actually suspect (won't know until I try) that the increased size of the new touchpad will be an important factor - you'll essentially be able to use a finger like a mouse on a pad - <USER3> needing to necessarily resposition very much - giving you veyr mouse-like behavior.

</comment></comment><comment><USER59> How do you right-click?

<comment><USER60> If its like the laptops you press down with 2 fingers at once.

</comment></comment><comment><USER61> 10/GUI just became a little more possible today.

</comment><comment><USER62> I've got a MBP hooked up to an external monitor, and I find myself using the four-finger upward-swipe to show my desktop.

That's about it.

</comment><comment><USER63> One word: Finally.

</comment><comment><USER16> This is a great idea, it's one of the reasons I wasn't prepared to get a desktop over a laptop (besides the obvious portability).

I think in the future there will be more integration of something like the iPad or the iPhone as direct interface to the computer.

</comment><comment><USER64> Now Apple just needs to incorporate this into the "iPad Keyboard Dock" and I'll be more than happy to buy!

<comment><USER65> How would that work? There's no cursor for it to control on the iPad.

<comment><USER66> It _could_ work. They could add a simulator-esque cursor without too much technical trouble.

That said, it _shouldn't_. Why would anyone prefer indirect manipulation of a cursor over direct manipulation of the screen elements? If you've got an iPad in a keyboard dock, you're primarily writing and occasionally navigating. I don't see the use case.

Gorilla-arm would only set in if you were primarily navigating, in which case you'd just undock the thing.

</comment></comment></comment><comment><USER67> I read this and thought this was a joke.

</comment><comment><USER42> I WANT THIS! ;)

</comment><comment><USER68> So, does the bluetooth disconnect when you accidentally touch the bottom left side with your thumb?

</comment><comment><USER69> This looks like a nice product and I do really love the multitouch pad on the MacBooks, but I'm so sick of Apple calling everything "Magic" and "Revolutionary".

</comment><comment><USER70> It should have been a mouse pad you can touch. Probably with a different design for mouse (you just need a metal point instead of trackball/optics).

<comment><USER55> You can use your finger....</comment></comment>


J'arrive - A easy way to share your location (subscribe beta) - <AUTHOR> http://beta.jarriveapp.com/

<comment><USER2> Admittedly, my French may be a bit rusty here, but what's the difference to, e.g., FourSquare or Facebook's Check Ins?

<comment><AUTHOR> @th0br0 J'arrive is absolutely not a check-in like app. It's an app that going to help you to share your location with your friends for a given amount of time. You can compare it with 'Find my Friend' but available for everyone (not only iPhone's user). And don't worry about your French, J'arrive will be available in english (with of course some french touch magic in it ;)).

</comment></comment><comment><USER3> looks great</comment>


Why do traders in investment banks feel their jobs are immune from AI, etc? - <AUTHOR> https://www.quora.com/Why-do-traders-in-investment-banks-feel-their-jobs-are-immune-from-AI-automation-and-deep-learning <comment><USER5> because traders are used to seeing such predictions fail.

Reuters was trading FX electronically since the early 1990s. At the tier one IB I worked for the IT budget was 500m USD a year (across products), and that was in 1997! Huge resources were thrown at automation. However, to this day, large trades in FX (> 10m USD notional) are still almost exclusively performed by humans over a telephone or over the bloomberg messaging system.

That's because, no matter how much you automate stuff, there is still the 1% "edge case" scenario where something goes wrong, and when that happens, you most definitely want a human that you can "look in the eye", when you have that sort of execution risk. Remember that markets move really fast and there is a lot of risk in big trades that "go wrong" because unwinding said trade will almost certainly cost one of the sides a fortune.

Also, high finance is not just about what you know. It's inevitably about _who_ you know, about "illogical" factors such as salesperson charisma, entertainment, and most importantly, a credible personality type that understands the edge case risks. These things are very hard to replicate with a machine. You'll say they should be, that these things are unfair, but they remain a fact after many attempts at removing them have failed.

As for AI, let's for now call it what it is: machine learning. Learning from the past. That's fine for recognising stop signs at different distances, angles and degrees of noise. But in finance, the past is often misleading. Sure there's trend, but there are also very big instabilities in the historical correlation matrix. Paradigms shift without you even realising it. The constant is change. AI is not good enough at that, _yet_.

BTW, that's not to say machines are not making inroads. It's becoming almost impossible to get a decent trading job now with knowing at least R and Python to a comfortable degree, and good quant programmers cost a fortune. There's massive demand.

<comment><USER6> Machine learning is "learning from data." It is not the assumption that there are no dynamics, and that the future will simply be a repetition of the past. To the extent that the future is predictable, learning from data is the best that can be done.

The reality is that speech recognition, language translation, face recognition, object classification and detection, semantic segmentation, speech and image synthesis have improved by extraordinary leaps and bounds in recent years. If we used the logic that past failures inform a confident belief that future success in a challenge will inevitably fail, then we should've bet heavily against Alpha Go defeating one of the most accomplished human Go champions in the world. Self driving cars seem like a sci-fi fantasy until they become a mundane reality.

There's an irrational arrogance to human beings in general, and Wall Street types in particular, regarding the specialness / non-reproducibility of their intelligence. It's not unlike the belief that people had that organic molecules were somehow special, "vital," and not synthesizable from base elements.

Certainly, there's a long way to go to replicate the capabilities of a human brain, but I don't think we should exaggerate or fetishisize the human power to estimate and mitigate risk. We've seen many spectacular failures of that in recent years.

<comment><USER5> The difference is that chess, go, etc are all essentially rules based. Finance has very few rules that do not break over time. Just look at QE. Arguably the financial market represents the collective intelligence of a huge amount of very clever people. Machines are only just starting to challenge a single human at a rules-based activity. We're very far from beating a brutally darwinian, impressively adaptive, human hive-mind whose main skill is figuring out when rules are about to get broken.

<comment><USER6> What are the rules for image / video captioning? For natural sounding speech synthesis? For realistic image generation? For semantic segmentation? For determining perceptual visual similarity between images?

Frankly, rules based AI is basically a bust compared to learning from data approaches.

The hive was pretty amazing at destabilizing the entire global economy within a brief span following fundamental financial de-regulation.

People flatter themselves in ways which satisfies their egos and self- interest. Of course they want to believe in, and have everyone else credit, their magical superpowers. No government (backstop) wires necessary.

<comment><USER7> Automatic image and video captioning is in a very sorry state. Machines can't do it.

Machine learning is rule based. The difference is just that it creates many more rules for more and more granular cases. But it will never be intelligent, it will always be a dumb digital bureaucrat.

<comment> yjftsjthsd-h Given results from ex. neural networks, ML "rules" appear to be granular and flexible enough that I'm not sure they usefully count as being rules anymore.

<comment><USER8> There are rules in that you're able to create an exhaustive labeled training dataset. The rules may not be generic, but they exist through a specification by example.

Beating a human when there is no discoverability to the problem, and when building an exhaustive dataset is a problem in itself is not on the horizon yet.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> A lot of traders are losing their jobs, and many fear this.

As other mention, though, Wall Street makes a lot of money trading the edge cases. For instance, many people thought derivatives traders would become obsolete when the Black Scholes formula arrived. In reality, the model grew the size of the derivatives market, and traders made money knowing where the model was wrong. (Example: It assumes constant volatility)

Similarly, many investors use an OAS (Option Adjusted Spread) model to justify prices on one-off Mortgage Backed Securities. This also helps grow the market, as there's more transparent pricing. But traders know where the models are wrong, and make money off of them.

When technology enabled FX trade spreads to be less than a penny, people thought traders were done. But this increased the volume of trading (more hedging became cost-efficient) so while the % skimmed by traders decreased, the absolute $s increased.

Net, as long as the financial pie grows, traders can find ways to siphon money off. That amount may grow or shrink, but generally the story is more technology has helped them.

Perhaps the best analogy is a chess expert paired with a computer can beat either the computer or the expert alone.

<comment><USER10> > Perhaps the best analogy is a chess expert paired with a computer can beat > either the computer or the expert alone.

This has stopped being true for a number of years. Computers play chess so much better now, that a human will actually impede it.

Think this way: could a 12 year old (the human) help a math graduate (the computer) on some problem? Or more likely he will just be a distraction?

More elaborations on this from gwern: [https://www.gwern.net/Notes#advanced- chess-obituary](https://www.gwern.net/Notes#advanced-chess-obituary)

<comment><USER11> A better analogy would be computer-assisted poker. The computer can keep track of the cards and provide probabilities, while the human provides the social computations (figuring out exploits by modelling the opponent's mental state). Computers have been very successful at pure optimization problems like chess, but still lag behind on games of assymetrical information.

The stock market is more like poker than like chess, because it is the correct application of theory of mind (Can a computer bluff? Can a computer cheat?) that elevates it above a game of chance.

</comment></comment></comment><comment><USER12> For around eight years my primary job function was to put investment bank traders out of a job, by automating what they did.

There were still humans in charge of the algorithms, but they moved more towards Python programmers than market traders.

Many of the "old-style" traders bitched about what we did, and most moved jobs to banks that were less advanced.

(I was in the interest rates line; typical trade size is $10M)

<comment><USER13> I am curious about how the old-style traders actually did their jobs. Did they base their trading decisions on data or instinct?

<comment><USER14> Trading illiquid products is all about knowing your market, and having a good idea of who you will be able to sell something when you accept to buy it (you are not in the business of taking a position you cannot get out of). Having a good understanding of the flows, who is buying, who is selling, who still has room for this exposure, etc.

This is achieved by discussing with sales people, who themselves discuss with investors, as well as traders at other banks (through brokers).

Now the definition of illiquid varies. Many products that used to be illiquid are now pretty liquid (interest rate derivatives) while many other are liquid only for small trade size (certain bonds). FX is an interesting example. It is very liquid but it is also a market where many clients need to make jumbo transactions on the spot, and these would be market moving if not carefully managed. That's something that could be probably automated.

I find the article very poorly written. "Investment bankers" is extremely vague and the breadth of very different product in different markets traded by investment banks makes this sort of generalization a bit absurd.

The other reason why I think the article doesn't make sense is that investment banks cater for certain giant markets (equity, FX, treasuries, etc) but also for a huge variety of niche markets, in which only a handful of banks and traders are active. If you add the salaries of the couple of traders in each of the active banks on one particular market, the savings you would do automating the market wouldn't justify the years of IT development to train and fine tune an algo, which would still need to be maintained as the market evolves (and then you end up overpaying an AI expert instead of overpaying a couple of traders...).

I think AI will shine at solving wide problems that affect a large number of people. Self driving cars. Butler robots. Building a house. Manufacturing something common. These markets have the scale to justify large investments.

I very much doubt that AI will replace every single complex task done by a man today, for the very same reason that software hasn't replaced every single manual task done by a man today: the cost of developing and maintaining software can easily exceed the salary of the few guys you are trying to replace. To overcome that with software, we need to dramatically reduce the cost of developing software, enabling ordinary employees to develop their own software. But even today we are very far from that. What's the % of a generation who can actually code out of college today? How easy and useful are the main programming languages? I'd argue we are now going into the opposite direction. Microsoft is poised to take VBA out of office. All major OS are evolving toward the iOS-style locked down platform where you can only run Apple/Microsoft approved software. Corporate IT is ever more locking down platforms with software whitelisting, etc. I wonder if the golden age of productivity improvement through software has not peaked.

<comment><USER15> >To overcome that with software, we need to dramatically reduce the cost of developing software, enabling ordinary employees to develop their own software.

I think this is the key to running an effective organisation.

It requires that most people know how to program, or at least know enough to understand what is possible to program.

I also don't see this coming, partly because the education systems are not really up to par, but also because it's really hard to develop complex systems.

I'd argue that the languages are not really the issue here even if the trends seems to go towards even crappier languages (like js, php) to really mess up the heads of beginners.

But even with a hypothetical "perfect language" the main problems is the huge amount of time and effort it takes to learn any programming language at all, plus the even more enormous amount of time it takes to write something useful even for a really experienced developer.

But sure. It's perhaps time for a new cycle of tearing down the "mainframes" (in someone else's basement this time) and reinvent personal computing for the 2nd or 3rd time...

</comment></comment></comment></comment><comment><USER16> There seems to be some confusion going on about investment bankers and traders in the discussion.

Trading has been changing significantly since the 'big bang' when trading went from pits to electronic. From there on in you see the evolution of algorithm / program trading. This area has been using quants for decades at this point. There are a good few big names brands out there that are known for being 'algorithmic heavy', Man, Citadel, DE Shaw come to mind (I"m a few years out of date). That whole field has been open to introducing automation / algorithms to create a business edge and will probably continue to advance because its good for business. The profile of traders has also changed (Barrow boys versus PhDs)

Then I guess on the other side is investment banking such as m&a, equity and debt capital markets. Generally there its relationship based , juniors work on pitch books which from what I saw / heard were generally overlooked. This is potentially a lot harder to automate away. Then the bank would try to pull in some rain makers or grow them internally to land big deals. Usually these opportunities open up because their clients (Other companies) have learnt to trust the organization or at the least learn to expect a certain behavriour when enlisting their services.

<comment><USER14> Agree. But even the trading you are referring to is the trading of liquid products (essentially equity). A lot of OTC trading is still very illiquid and will likely not move to electronic platforms for the foreseeable future.

</comment></comment><comment><USER17> I've got a story that connects the new and the old way of trading.

When I joined the industry just after the millenium, I joined a firm of guys who used to be floor traders. Basically the guys from "Trading Places" with Eddie Murphy: coloured coats, loud shouting, eating contests. As London got automated, they moved "upstairs", which basically meant holding the eating contests in a room of screens and squawk boxes. It was a fun time (but not for everyone; old school also means macho culture and sexual harassment lawsuits). One day I thought to myself "in what other job in the world would you find your boss breakdancing?"

I checked up on that breakdancing guy the other day. He's continued along the way of old school market makers, taking calls from brokers and manually entring them into a system. He literally said "Nacho, I'm a dinosaur. I can't code, but I have 25 years of experience trading. And trading changed. Everything is eaten by the computers, and on top of that we have free money keeping the market from having more than one opinion."

We talked about a guy who used to work in the firm we were at. He'd gone the other way, and caught the start of the HFT boom. Now he's a billionaire. It's amazing that someone could go from the pit to trading several percent of global daily volume each day.

But basically, the old school traders are well aware of what the computers can do. They aren't stupid, the ones who can't code know they can't, and they can see the writing on the walls.

As far as I can tell, there's only one area of trading that's relatively immune to the machines. And even that isn't completely immune. It's special situations. That's where you're looking at corporate events like mergers, rights issues, and so on. It's somewhat hard to automate because there's just not that many things to make bets on. One guy can sit and read through a bunch of events and put on big bets, and there aren't that many people who have the specifics of how to make the decisions. So the benefits of automating it are not as huge as with most other types of trading.

</comment><comment><USER18> The question is a logical fallacy. I worked at an IB and even in 2011 traders were acutely aware of the benefits of technology and eager to invest in it and embrace it. Many of the answers here are regurgitations of techno-utopian talking points that do not take into account three issues

1\. Traders generally exert an advisory/supervisory role on the set of prices that the bank offers, prices which are based on a formula or other fairly automatic means, with an added human adjustment. They already extensively use technology.

2\. Traders are therefore most involved where profit can be made but simple algorithms don't work. For example, pricing big deals in illiquid markets, like when a company issues a large complex bond. As this contract is by definition not traded yet and not the same as others, there is necessarily limited applicable training data, so that there is no way to learn by example - i.e., use deep learning techniques (what I assume the question is asking about). In this case, trust and relationships are extremely important as both sides of the deal have limited information.

3\. Markets change dynamics, often very rapidly. Traders have to react intelligently to events: like interest rates hitting the zero lower bound, wars breaking out or industrial accidents. They need to anticipate the actual consequence to future cash flows and also to sense the appetite of the market after the event. Publicly announced AI techniques are very far away from this kind of complex general reasoning.

The days of manual trading are long gone: of open outcry traders, yelling in bullpits and making handsignals, when banks would hire big imposing ex- football players. The question is a "why do you feel you can get away with beating your wife"?

</comment><comment><USER19> The last few minutes of this documentary on Long Term Capital Management and its algorithmic failure will tell you why: [https://vimeo.com/28554862](https://vimeo.com/28554862)

<comment><USER20> The complete lack of any sign of remorse on their faces about having burned a trillion dollars made the hair on the back of my neck stand up. I've never seen such a blatant display of psychopathy.

</comment></comment><comment><USER21> Due diligence on the financials of a company (what investment bankers are supposed to do) is actually really hard to get right with the algorithms we have today. Much of the data and insight compiled by an I-banker today does not exist in an easily parse-able form for automated algorithms, and a substantial amount of the computation relies on common sense knowledge.

<comment><USER22> > _Due diligence on the financials of a company (what investment bankers are > supposed to do) is actually really hard to get right_

Diligence is automatable, in the long run. There is a human element to it, but it is small in most contexts.

Bankers solve trust problems. An AI would need to be trustworthy in a personal way to replace bankers--this only happens with AGI. For a long time, as long as humans control capital, there will be other humans interfacing those humans with each other.

<comment><USER23> > An AI would need to be trustworthy in a personal way to replace bankers-- > this only happens with AGI.

This happens with an unbeatable public track record. We trust AIs in lots of tasks already, and that is not because they smile and speech nicely.

</comment></comment></comment><comment><USER24> I generally don't think that all traders believe their jobs are immune. Any rational person that looks back at history will observe that the markets have constantly been changing due to technology that speeds up a trade or makes it more accurate. However, most of the significant changes have not been to due to Artificial Intelligence innovations. Instead, those changes have been about automating tasks that were once repetitive. I wouldn't necessarily call that AI.

If you'll allow me to simplify the IB Trader's job: there are 2 types of traders: agency and principal traders. Agency traders build relationships with clients, accept orders from them, execute them in the market. They make money on commissions. Principal traders will take risk. Sometimes on behalf of a client - on the back of a client order - or sometimes purely for the bank's own account.

The agency traders have seen their roles decimated by technology - <USER2> much of their role (minus the relationship building part) was automatized. On the risk-taking side, AI has crept into some places - albeit in very narrow use-cases. For example, we've seen the rise of the robo-advisor, where an "algorithm" comes in and automatically adjusts your portfolio to reduce risk and increase alpha. Well, the risk reduction party is well-known (Markowitz portfolio theory). But the increasing alpha part is the difficult thing. And AI seems to be quite far off in its ability to be a stock picker - <USER3> because the passive approach is superior (ie, no intelligence needed at all).

</comment><comment><USER25> Because investment bankers are just salespeople? Nobody is suggesting AI will replace salespeople in the near future. I could see there being fewer grunt analysts in the future, though.

<comment><USER26> > Nobody is suggesting AI will replace salespeople in the near future.

They aren't? Because I see this happening all the time. Automated checkout lines, E-Trade and online brokerages, etc. Low-value transactions are handled more and more by electronic "salespeople". All it takes to reach high-value transactions as well is for the clientele to decide they trust machines more than humans, which is not all that unlikely given the many sources of human error involved.

It's possible that humans will be involved in high-value transactions like investment for a long time, since the marginal cost of human labor is low as the numbers get bigger, but it's not impossible to replace; we already have the technology.

<comment><USER27> I think that the term 'salespeople' here refer to those who proactively push their products to the market and by doing so widen the market for the company they represent.

The examples you give are indeed places where buying or selling happens, but where does the incentive to do these transactions come from? It's still from humans. Sure, we are bombarded left and right with ads that were carefully targeted for us by complex algorithms, but I don't think I've ever bought anything from Amazon without reading a review about the product from another user, or bought stock without going through several opinions written by more- or-less respected analysts.

</comment></comment></comment><comment><USER28> > "Would you trust purchasing a house from a seller, without meeting/talking > to them, or a single person before and throughout the purchase?"

You mean I can get an unbiased look at a house in peace, compare the numbers, look at the plans, measure the humidity and do my due diligence without a sales person breathing down my neck?

Hell yea. I'd pay premium for that.

<comment><USER29> This is the classical hn bias. Normal people would like to talk to humans. Hn users would rather get info, plans, etc from houses and eventually buy them and sign the contract using a REST API with a node.js client..

<comment><USER30> That's a rational buyer...what do you think the bank does? They run the numbers to see if the person qualifies, has great credit, etc...do you think they care how your day is going? Lol...

<comment><USER29> Of course they don't... But they are not the client... It's the client who needs honeyed words, and that an AI can't do..

</comment></comment></comment></comment><comment><USER31> In an investment bank a trader has a high ratio of support staff around them: legal, compliance, IT, operations, quants, finance/tax, risk. These 'support' jobs are a significant fraction of the real cost of a trading seat - not just the trader's salary.

Automation has happened incrementally in the industry for years, like many others - starting with the easiest stuff (low hanging fruit) like some operations tasks and mechanical trading tasks, and leaving the more complex tasks for humans - or letting a human scale to do more.

The more complex tasks that are left typically require non-trivial intelligence, e.g. understanding why the new product brought to market by your competitor or counterparty is slightly different to what you are trading today, and deciding if you can/should transact in it. Understanding what impact the upcoming compliance rule changes have on your market and activities (there are always regulatory rule changes). Understanding what the limits of your trading are, to avoid concentrating too much exposure in one area. Understanding why your counterparty is upset about some aspect of the transaction. etc.

</comment><comment><USER32> Famous last words for a lot of businesses:

>When it comes to AI/Machine Learning: the nature of ____ and markets are drastically different to other fields that AI/ML have previously excelled in. The main reason being what are the laws and rules that govern how an AI/ML should view a field?

>Since the very nature of a market is… a constant change respecting an infinite and broad amount of variables (____), a complex system (repeating exact past actions, will not give you the same results) and a social interaction/conflict regarding an ambiguous ____, it becomes incredibly difficult to determine those laws and rules.

>In physics, mathematics, computer programming, you will always have a safe level of predictability when it comes to certain functions and how they interact to give you a reliable solution. With ____, instead of this safe level, there is just irrationality beliefs; something you cannot model with accurate certainty (just look at how AI/ML would handle predictions for Brexit and Trump, and then, how it would cope with the aftermath).

<comment><USER33> Markets predictions were at 20% of Brexit and Trump the day before the vote. And moving from 15 to 50% during the week before that. Noting they followed similar patterns.

They assessed the risk fine.

</comment></comment><comment><USER34> _a constant change respecting an infinite and broad amount of variables (macroeconomic), a complex system (repeating exact past actions, will not give you the same results) and a social interaction /conflict regarding an ambiguous valuation of a price, it becomes incredibly difficult to determine those laws and rules._

Just like driving a car.

<comment><USER35> Well, no, not really. Ideally a car has a definable spec, such as a road with consistent markers. Steering within a consistent set of lines and reacting to other cars on the road is difficult enough, especially if you try to account for random behavior from other drivers.

Modeling and forecasting a security's price mathematically, or a set of security prices, is complex in a different sort of way. A better analogy would be like trying to program a car to deal with a meteorite that suddenly hits the road in a random spot, or a bomb that goes off 100 feet in front of the car as it's cruising on the road. There are far fewer rules with which to start a foundation, and the entire macroeconomic structure is far more delicate and in constant flux from news, other traders competing against you, information closed to the market, etc. With cars, you at least get a road, and add randomness in with other drivers. With security prices, you _might_ have a lower bound of potential losses, if you're buying long. It's just a different animal.

</comment></comment><comment><USER36> Maybe with their work, labor costs are a relatively small portion of the product price?

<comment><USER37> Look up the comp ratios of some of the major investment banks. It can easily be 50%. As with all professional services firms, the biggest costs are people and real estate.

<comment><USER38> I think he's saying the money client pay to investment banks is a small compared to size of the deals bankers advice / underwrite etc.

<comment><USER37> But it is large compared to what the shareholders earn, and they are the ones ultimately pulling the strings. The days of the white-shoe partnership are long gone...

</comment></comment></comment></comment><comment><USER39> If traders have managed to persuade people that they are able to beat the market against all evidence[1], why would they not be able to convince people that they are better than AI?

[1] Of course, there are anecdotal evidence of the opposite, but there _should_ be those if the performance of a trader is a random process.

<comment><USER35> Do you consider several hedge funds consistently beating the market with significant margins for over 20 years to be "anecdotal" evidence?

The evidence shows that it is very difficult for traders to consistently beat the market. The evidence does not show that their performance, as a profession, is random.

<comment><USER39> Yes, I consider them anecdotal. If you have a billion people flipping coins, a few are for sure getting 20 tails in row, but that is no evidence that they would be better coin flippers than the others.

> The evidence does not show that their performance, as a profession, is > random.

I think you are right, if I recall correctly, the evidence points to a conclusion that the performance of their profession is _worse_ than random when you take fees into account...

<comment><USER35> Yes, that's the classical coin flipping example from the strong position on Fama's Efficient Market Hypothesis. There are several problems with the coin- flipping analogy:

1\. As stated, it's not falsifiable. So you start with a conception of the market as entirely random, and you observe that participants are consistently beating this market. Each time you observe someone beating the market, you chalk it up to the probability distribution. _" Well, that's just a two-sigma event._" Then you see it happen again. _" Well, that's just a three-sigma event._" Then again, and again, and again. How many sigmas from the average market performance are you willing to accept before you agree that someone is legitimately and purposely beating the market with a skill-based mechanism, not a chance-based mechanism?

Furthermore, do you have the numbers to turn this into a falsifiable claim? What is your time interval? Daily, weekly, monthly or annually? How many correct forecasts do they have to make ("how many sigmas from the average"), compared to the chance expectation of coin flipping over the same timescale? If you don't have these numbers handy, then it's purely a thought experiment. Subsequently, the observation that funds like Berkshire Hathaway, Bridgewater, Renaissance Technologies, Baupost Group, Citadel, DE Shaw, etc. consistently beat the market for at least 20% _net of fees_ over 20-30 years suggests that, per Occam's Razor, people can beat the market due to skill.

2\. The analogy is not comparable to active trading. You don't need to hit 20 heads in a row to beat the market consistently, you just need to hit a p-value number of _x_ heads correct for _y_ coin flips greater than chance would suggest. We don't assume that basketball is a game of chance if the players can't make all their shots in a row; nor do we assume that baseball players with a 0.3 batting average aren't clearly better than the average high school dugout. If your trading interval is weekly or monthly, and you're consistently up over the market (even net of fees!) for 240 months or 360 months, it doesn't matter if every single month was a winner.

3\. Have you ever read Warren Buffet's response to the EMH assertion, as postulated by Fama?[1] He outlined an excellent rebuttal in his 1984 _The Superinvestors of Graham and Doddsville._ Essentially, if you assume that the coin flipping analogy _does_ map to trading, then you should expect to see a normal distribution of the winners, given that the market is inherently random and no one is achieving superior coin flips through skill. However, if you observe that the winning coin-flippers consistently hail from a small village with standard coin-flipping training, then it is more reasonable to assume that there is something unique about those particular flippers. This is what we see in reality - yes, most amateur traders fail miserably, and yes, most hedge funds underperform the market over time. But there is a relatively small concentration of extremely successful funds and traders in an uneven distribution.

4\. Even Fama has walked back on Efficient Market Hypothesis, and no longer espouses the view that the market is inherently random. It is deeply complex, yes, but it is not efficient, nor entirely random. Several studies have been conducted to empirically examine EMH, and the results in favor of the hypothesis are dubious.[2][3][4] A much more charitable retelling of EMH is the weak position, which essentially states that any _obvious_ alpha will be quickly arb'd out of real utility, but that _non-obvious_ alpha, or alpha which is technically public but not easily _accessible_ will retain utility until it becomes obvious. This also maps more cleanly to reality, in which trading on e.g. news reports is mostly unprofitable (everyone can get a news report at around the same time, for the same level of skill) whereas mathematically modeling pricing relationships can be extremely profitable (doing so accurately requires public, but mostly unclean data and a great deal of skill).

_______________________________________

1\. _The Superinvestors of Graham and Doddsville_ \- [http://www8.gsb.columbia.edu/rtfiles/cbs/hermes/Buffett1984....](http://www8.gsb.columbia.edu/rtfiles/cbs/hermes/Buffett1984.pdf)

2\. _Investment Performance of Common Stocks in Relation to Their Price- Earning Ratios: A Test of the Efficient Market Hypothesis_ \- [http://onlinelibrary.wiley.com/doi/10.1111/j.1540-6261.1977....](http://onlinelibrary.wiley.com/doi/10.1111/j.1540-6261.1977.tb01979.x/abstract;jsessionid=93367F17F01A240FFD25E0CB0BF66FB8.f03t04)

3\. _The Cross-Section of Expected Stock Returns_ \- [http://onlinelibrary.wiley.com/doi/10.1111/j.1540-6261.1992....](http://onlinelibrary.wiley.com/doi/10.1111/j.1540-6261.1992.tb04398.x/abstract)

4\. _International Stock Market Efficiency and Integration: A Study of 18 Nations_ \- [http://onlinelibrary.wiley.com/doi/10.1111/1468-5957.00134/a...](http://onlinelibrary.wiley.com/doi/10.1111/1468-5957.00134/abstract)

<comment><USER39> My belief in some of the weaker forms of EMH does not stem from the idea that markets would be somehow correct, but vice versa. Markets are (almost[1]) always incorrect, and nobody can know how much they are incorrect tomorrow[2]. Thus nobody can beat the markets, other than the random coin tosser.

Now, I fully agree that there have been certain anomalies (value premium anomaly in case of Buffett) that pretty much align with the strategies of these long term successful investors. Question is, has it been their skill to pick right anomaly as a basis for their strategy, or luck? Again, in the world of investment strategies, there for sure is _someone_ trying almost anything. And if that anomaly disappears[3], do they have the skill to change their strategy?

But we are a bit off topic here. The original question was "Why do traders in investment banks..." that is a different species from the warrenbuffetts.

[1] Asset markets can be right somewhat like a clock that has stopped is right twice a day.

[2] Yes, Keynes said "The market can stay irrational longer than you can stay solvent."

[3] I have actually bet my money that the value premium anomaly is not disappearing, but that anomaly is not something investment bank traders can enjoy, as the anomaly is far too long term anomaly for them.

</comment></comment></comment></comment></comment><comment><USER40> Is there a fallacy name for this? i.e., asking a question that suggests something ("investment bankers have this feeling") as a premise, that may be not true.

<comment><USER41> It isn't a fallacy to ask a question :)

<comment><USER42> Indeed it can be: [https://en.wikipedia.org/wiki/Loaded_question](https://en.wikipedia.org/wiki/Loaded_question)

</comment></comment></comment><comment><USER43> For similar reasons that spreadsheets didn't put accountants out of work.

</comment><comment><USER44> Honestly, maybe they just don't care? Rather, what should they do about it, what would we expect people with such huge earning potential _right now_ to do other than push forward with the plan that works under the status quo.

Answering my own question: I'd expect bankers to save more (of their own money) in anticipation of the good times not lasting as long. More conservative types will weather the storm and spendthrifts will get wiped out.

In other words: business as usual, up until the very moment it isn't.

</comment><comment><USER45> Everybody feels _their job_ is safe from automation. Its the same way planes crash but _not mine_.

Believing "I am special", is just built into us.

<comment><USER46> Not only do I believe that my job can be automated - I've spent years trying to do just that.

Of course, I wouldn't be giving the scripts to my employer, just spending more time doing other things...

<comment><USER45> I think the difference here is "task" !== "job". You're automating tasks. Your job is to keep the system running as effectively as possible (I don't know what you actually do). Meanwhile, I do get that you're not actively trying to lose the responsibility of these tasks, i.e. "I wouldn't be giving the scripts to my employer", but that is because you're trying to hold onto the relaxed transition phase between tasks X, Y, Z to tasks 1, 2, 3. If you really had job security issues, you would not have written those scripts.

Are you sure you haven't perhaps fallen prey to too much job security? Such a large sense of job security that you don't care about automating X part of it away. You, like me, like other "rockstar" software developers, (perhaps[0]) believe it doesn't matter if you automate X, Y, Z, because you're so good, that when you're out of responsibilities you will be offered the followup/orthogonal tasks 1, 2, 3.

Anyways, I'm just trying to highlight that you too, perhaps subconsciously, think "I am special, it wont happen to me", because at least I who also don't care about automation feel this way.

[0] I know for me, this is the truth. This overconfidence is why I don't care about automating my tasks away. Honestly, even giving the scripts to my employer makes little difference to me.

<comment><USER46> To some extent what you say is true, but only partly.

I'm not part of the cult of the "rockstar programmer". I do write a lot of good code, and I tend to do it much faster than other programmers that I work with. But programming is not actually my job description.

Without going into details - I do something else for living, that uses computers quite heavily, and automating my job is more of a hobby. I have a huge amount of flexibility in what I do, so there is an element of substituting 1,2,3 for X,Y,Z because I find them more interesting. My boss wouldn't care one way or another - he is more interested in whether or not the tasks get finished, and maintaining a high quality level in the final product.

</comment></comment></comment></comment><comment><USER47> A good trader can explain their trades, a good AI cannot _yet_. This is important in many financial settings when some human must still be personally responsible for the trades. For a manager it is much easier to approve regression coefficients or a particular trading formula than to sign a neural network into production.

<comment><USER48> _> A good trader can explain their trades_

Yeah, but what are the odds that their explanation is just a post-hoc rationalization (ie. "heads I get credit, tails I avoid blame")?

</comment></comment><comment><USER49> what a strange question. technology has eliminated swathes of traders, open outcry exchanges, equity execution traders. if you watch CNBC, you see almost no one on the floor of the NYSE , because there is nothing worth doing there. (except for a couple of exceptions due to exchange rules)

</comment><comment><USER50> Some vocabulary might be helpful here, generally there a few buckets most traders can be put into:

1\. Proprietary trading (think hedge funds or the old bank prop desks, ranges from the bass brothers to renaissance tech in terms of style)

2\. Execution or sales traders (often work for or are the counterparts to #1, their goal is get the best price for their clients in the quantities and time frames desired)

3\. Market making (essentially providing liquidity to both buyers and sellers)

4\. Sales - essentially layered on top of 2 & 3 to interact with 1

With that said, #3 has pretty much been automated in some markets and is the easiest to automate, #2 has had a lot of automation for vanilla stuff, but people are still involved in big, complicated things. #1 may be automated but is largely strategy dependent, #4 is pretty hard to automate...

</comment><comment><USER51> I don't agree with the question, especially if we are talking about de facto traders. There is a high level of worry because commission margins and returns per unit of risk are the lowest ever. I would say there is a high level of calculated ignorance in the industry.

Truth is, most trader jobs have been gone in the last 15 years. First it was the automation of order collection (phone orders being replaced by automatic order routing processes), then automation of the execution process (from human order management to algorithmic trading), then automation of the investment management process (from discretionary processes to quantitative management), and the last frontier will be the automation of the relationship with the client.

Problems are different for different trader types.

Execution traders execute orders on behalf of clients. Today a lot of clients execute directly using web terminals and APIs. And if you still execute on behalf of others you spend a lot of time fighting against the machines (one solution is to trade out of market, but this too is being automated). Very very hard to come up with new execution algos.

For delta hedging traders the role has been reduced to monitoring, unless you have allowance to take risk (which most banks don't since the 2008 crisis).

For alpha traders, whose job is to extract money from market, the risk is not in being replaced by the machines, but one of overcrowding, i.e. everyone following the same strategies is reducing returns and increasing risk (skew) across the board.

Sales traders whose activity is to sell first, and trade last, still need to manage relationships with customers. But this has been changing gradually, with clients prefering digital interfaces, passive investment programs, and now the rise of automated investment management programs.

I started working in the industry circa 1999. The head of trading I was working with told me once machines will never be able to invest like a human. He's still there doing a lot of money, but certain that it will end someday. His strategy: "let's see if I can make it one more year"...

</comment><comment><USER52> If the role of investment banking is to optimally allocate capital, then part of that job is research. Think Andrew Left's exposing fraudulent Chinese tech stocks, or the Lumber Liquidators controversy. Algorithms can augment this work, but cannot replace it.

</comment><comment><USER53> Well, an equally good question is why do programmers feel their jobs are immune from AI, etc.?

<comment><USER54> Interesting, I never thought about that. It seems like something very far off, though. Does anyone know about any research currently being done in this field?

<comment><USER55> That's what (supervised) machine learning and neural networks do, isn't it. You specify the "what" and not the "how", they figure out the "how". Basically techniques for automating the task of computer programming given requirements.

So any advance in AI/ML/NNs can be seen as advances in the field of automating computer programmers. We don't tend to think of it like that because these techniques have to be applied, currently, by computer programmers, and the software the techniques "write" was generally infeasible to write by hand anyway: it's been so far purely additive and solving problems that hand- written software either couldn't do at all or where it sucked and progress was very slow.

But watch out. The tech is getting better scarily fast. Neural networks have been able to learn how to do basic algorithms like sorting given only examples. The primary limit is still the amount of data they need to work with, but R&D is driving that down too.

I can see a time when many tasks that today automatically require skilled programmers are instead done by domain experts who simply shovel data into an advanced neural network and discover it gets results that are good enough. Perhaps hand-written software made by a skilled programmer would be better, but the machine is a lot cheaper ...

[http://www.i-programmer.info/news/105-artificial- intelligenc...](http://www.i-programmer.info/news/105-artificial- intelligence/7923-neural-turing-machines-learn-their-algorithms.html)

</comment></comment></comment><comment><USER56> Traders will still be needed to justify bad decisions, or 'good' logic with bad outcomes run by AI. ;)

Everytime I see a gameshow where you can 'bank' your current winnings, ... I imagine the future of trading will include some strategy that has an AI yelling those sorts of things to other ai agents acting in concert. The strategies of the macro stock market positive outcomes, being applied to micro stock actions. [https://www.t0.com](https://www.t0.com) is also going to be a fun reality.

</comment><comment><USER57> For one thing, traders have knowledge of multiple markets and real-world events (hurricanes, elections, etc.) can infer correlation between those and place trades accordingly. AI is completely unaware of these things and can only train itself on the market data it trades against.

Also, not all trading is liquid, nor is it short term, both of which are targets for automated algorithmic trading. Long terms investments are typically human decisions.

</comment><comment><USER58> God I'm so fucking jealous of computer specialists who work in investment banks. How do I get in without going to Harvard?

<comment><USER59> They are exactly what they claim to be: specialists. You need to become one in some area that banks deem valuable. Current white-hot areas would be FPGA, machine learning (but you'll also want a PhD in statistics, or at the very least a good degree in it from a good uni) or be a badass systems developer who can write absurdly low latency code in terms of allocation, cache coherency, network sympathy and so on. To get this good, you need to have been doing it for a decade or more, so start now. At the bottom. No one leaves university and gets one of these jobs. They leave university, maybe get a PhD or work for a decade and work up to them. You are jealous of people who have put in thousands of hours of very hard work into their careers, but will you do the same?

<comment><USER12> Large IBs do hire dozens of new graduates every year into IT.

You have to be bright, articulate, interested.

<comment><USER58> That tends to be Java CRUD not trading logic.

</comment></comment></comment></comment><comment><USER60> > "Why do traders in investment banks feel their jobs are immune from AI, > etc?"

Why do people need to have anxiety and fear about AI is the better question. Let's solve those instead of this blindless ambition to automate everything, and then somehow assume our benevolent government will give everyone UBI or some other pipe dream.

</comment><comment><USER61> I just think that if AI could beat them, they were already replaced. Any innovation in trading is automatically implemented. May be this will be possible in the future but it doesn't depend only on deep learning techniques and having huge samples for learning because they have both.

<comment><USER62> AFAIK the bulk of investment bankers are beaten just by chance as this article suggests: [http://www.automaticfinances.com/monkey-stock- picking/](http://www.automaticfinances.com/monkey-stock-picking/)

<comment><USER31> The bulk of investment bankers are not picking stocks for funds. In the field of fund management, there are some people doing this, and it is a very common trend that passive indexes do better than the active funds - it is true.

In the field of investment banking trading (which is mostly market making) the amount of automation varies by asset class: very automated for some asset classes like fx, equities, much less for more illiquid asset classes like credit, commodities, bespoke products.

As well, senior traders operate as the 'business' making more decisions than pricing of products. They make the business decisions often judging legal, compliance, accounting risks. (and not always correctly.).

e.g. do you accept to trade with a Dutch counterparty who wants to trade against your German legal entity knowing that you can only hedge the position in London? What is the risk between the two legal setups? What premium should you charge for those risks?

Do you trade the very large size that the counterparty wants, knowing it takes you over your balance sheet position limit - can you get approval for this from your senior management? Can you offset the position in the market without it moving against you? What premium do you charge for this?

</comment></comment></comment><comment><USER63> I don't think bankers are naive, they are seeing it happen.

If your job can be done overseas, it probably will be outsourced.

If your job is algorithmic, you'll be replaced by a computer.

A lot of M&A activity though doesn't fit that category.

</comment><comment><USER64> If it's true, it could be that there is more insider information being used than is generally recognized. Computers can't get that info.

</comment><comment><USER65> They don't. I work in in the trading department of a large bank - directly in finance, not IT. Have been here 10 years.

Here's what I have seen. For more "liquid" markets - like delta-one blue chip equities, IR swaps, spot FX - jobs ARE moving quickly. People are not complacent. Competition comes from firms that are much faster in execution, having automated most of the process - and presumably having lower hedging costs by being able to move quickly. Or just basing their decisions "better" on history.

Having said that, you have to remember that: a) Size of balance sheet matters. A large bunch of clients of such firms trade with them due to "synergies". Large shops are one-stop shops. They don't just sell you a swap. Its usually a loan/funding + an option to hedge it + some tax advice thrown in. Its hard to break it down into components and trade individually, as most other players may not be even willing to face such counterparts. And shopping for each component separately might mean larger time and friction costs. Here being a large bank helps massively. Basically - capital is hard to be automated, decision making is easier.

b) There are a large number of markets that cannot be automated anytime soon. These are "illiquid" or "exotic" markets. The data on them is scarce and trades happen in a lot less frequency, then say swaps or delta-one equity. Throw in some options, and your model might say here's a price, but there's no guarantee on it. Then you have to balance it compared to your book size, and the price is not discoverable, or even same across firms (for good reason - <USER4> cost of cash liquidity might be different to yours). Sometimes counterparts will trade with you even if you have a worse price, because there's lower risk of you falling down in 10 years on a 20 year cross currency trade or inflation swap.

c) Good traders are very defensive and paranoid. They don't work under the paradigm that history will repeat itself. They will try to look for risks that might seem to have nothing to do with their market. I know some traders who have been making good money every year in these exotic markets, regardless of crisis or not. Some of the sharpest people I know - and they love their job. Not just the money aspect. They just love the whole high pressure environment and everything it entails. Now they are being drowned in regulation. Don't get me wrong - regulation is good, but right now we have a mess. I won't try to convince you - and unless you work in a bank, you will not agree with this.

d) Management is old. Might seem obvious, but most of the people at the top have made their money the old-style way. After a certain amount of time, you tend to trust your own instincts more than other things, especially if you've been successful (which may have just been a random process to start with). So they think this is just another fad and it will pass. These things have come and gone - we're still here. This is a prime reason why some banks fail suddenly. The top guys just didn't see it coming.

</comment><comment><USER66> Because investment banking is a sales job.

</comment><comment><USER67> Investment bank business is all about conncetion. Connections lead to deal. Machine can not replace that forever.

</comment><comment><USER68> Because investment banks are run by sales. Only the people at the bottom actually care about this stuff

</comment><comment><USER69> >No amount or greater sophistication of the algorithmic structures listed above, can replace genuine human nuance, interaction and trust.

I don't think this is true. Not at all.

<comment><USER70> Me neither. How much of the genuine human nuance is present in today's financial services or any commission driven industry - next to nothing. Every agent/broker is motivated by the highest commission he/she can make, nothing more.

<comment><USER8> I can reasonably say that you make this comment because you know nothing about the business logic of trading desks.

A trader is not necessarily a sales or a broker, though. Nor is he a quant, or a dev. People rarely imagine how many different jobs are involved in the trading job, and how rich the business logic is. At my shop, traders are the piece that connects all of the jobs in the value chain.

I believe, currently, the business logic can be improved locally by learning systems (and it is), but there is no public example of an industrial learning application encompassing a scope comparable to what the usual trading desk handles. Sure, there are many inefficiencies ; traders work on heuristics, afterall. But I don't believe we have the necessary horizon to aptly predict the end of traders, because I don't see how we could make AIs with a better efficiency.

<comment><USER70> I'd say I know something about the business logic of trading desks. Some of my friends worked at GS. I'm not implying traders are brokers. I do agree that the trading business logic is very rich and that AI will be nowhere close to replacing a trader in the next 15-20 years at least.

I do foresee a future in which a trader can accomplish a lot more than he/she can today using AI. So in the future as the per trader efficiency increases, the number of traders required will most probably decline, unless there is a dramatic increase in trading volume that cannot be matched by the then state of the art AI.

This is essentially what's happening today with X.ai, Facebook messenger and the like. Sure the logic involved in booking air tickets for a group of 5 over 10 conversations isn't as complicated as the rich business logic of a trade, but 5 years back Facebook messenger would've seemed almost impossible, just like the trading business logic seems impossible to do with AI today.

<comment><USER8> I have no doubt new technology will be leveraged to improve productivity, but that's been quite common in the last decades.

On the other hand, the technology advances needed to transform the tools into standalone actors are not merely a matter of scaling current technology. Especially, the creation of training datasets is a problem for which we currently have no solution, that's why we fall back on human trainers (mturk, etc). That's why a solution to a problem with no clear, bounded model and no easy dataset seems out of reach.

</comment></comment></comment></comment></comment><comment><USER56> Why do programmers think their jobs are immune from AI, hubris and history.

</comment><comment><USER71> This post will get buried in this discussion, but I actually worked for a HFT company trying to break into all sort of algorithmic trading (and made a small fortune doing so).

The topic question is:

1\. Loaded question

2\. Uninformed question, look at a small subset of the electronic trading _reality_.

3\. Overly broad: jobs are immune from AI, etc (what? depression? ancient aliens?)

So, in order:

1\. Some traders will always get fired as they can't do their jobs (or are just plain unlucky in the wrong time).

2\. The 'traders', or rather what people think of a 'trader' is a wrong image in what I worked with. They were people configuring algorithms, linking dependencies between tickers in various market segments, etc. These traders did some rudimentary AI guiding of a sort. They used their brains, read newspapers, tried to code engines to parse said newspapers with machine learning long before it became media catchphrase, failed (badly), configured various triggers. They weren't 'traders' in your ordinary sense of the word, but struggled to make that work. It takes a _lot_ more in scope, than buy low sell high' to make trading work in this millennia. Several magnitudes or more work, if I'm to scope it. It's a corporate effort, not a singular one. Traders do get fired but whole corporations go under also.

3\. The 'etc' part:

\- AI will never advance to the level to predict the future, as it will not know the sum of human experiences driving that future. This is especially true for the financial market as it is linked with the rest of all human activities in a un-linkable fashion. I can take this as a philosophy thought experiment, or as an economy fact, or from my limited experience, but, take it as you will, AI is a 2017 buzzword to come. Take it with a grain of 100 shares worth of salt.

\- There is a very _finite_ liquidity in the market, compared to the money available. When you have to iceberg orders to not destroy a trading strategy and move the gap for _everybody_ , it's really obvious that the money you can make are _limited_. This was true in 2010, when most of the major banks decided to also enter, and is more true today. So, connected with the previous point. AI will never be _fast_ enough, to earn on the limited liquidity compared to available players, considering we are capping off in silicon CPU processing power right now. I'm open to arguing this point, as it was argued to death among friends in the last decade, but I don't see this moving in any direction unless processing power moves off silicon, and even then, for a while. You can't scale atoms, and you still need 80 of them (or whatever is the latest) for building anything, so that's the hard wall. [extra thoughts] Given all of the above, humans have an edge, and humans with experience, have a sharper edge than the AI. In trading, nothing will change, and AI wont _replace_ traders. We can argue this point, feel free to reply.

\---

[extra thoughts] Purely my take on this, as future trends:

Javascript as popular web language will get capped off on performance soon (few years?). Thus, as demand increases one of these will happen (or all three of them):

\- Javascript will get threads, locks, and the rest of the circus and Javascript programmers will have to learn a lot more on top of what they know (in a language not quite friendly for writing threaded code in the first place) and it will be super fun to be one for a while.

\- A new language will emerge to replace the web languages and get more performance by simplifying the interaction with he browsers (Think CICS + IBM + inline assembly there)

\- The JavaScript+Browser Layers+OS+VM approach will get replaced with something more lean in the next 5-10 years because people will want more (whatever more is, nobody predicted it 100% so far).

Anyways, this post got unintentionally too long, I guess I had something to say on the subject :)

</comment><comment><USER72> Because what they are doing is really (an indirect form of) sales.

</comment><comment><USER73> A lot of them didn't understand technology.

</comment><comment><USER74> Insider trading can not be automated ;-)

</comment><comment><USER75> Reflexivity.

</comment><comment><USER76> can AI discover insider info?

</comment><comment><USER77> HN title is significantly different than the question the submission actually asks which is "why do TRADERS in investment banks . . . " and should be updated

<comment><USER78> In case the full title makes a difference it's "Why do traders in investment banks feel their jobs are immune from AI, automation, and deep learning?"

cc @dang

<comment><USER79> It looks like the question title was changed, in order to aggregate similar questions/answers. Most of the answers answer the question in the current title. I agree that trading and IB is a big distinction, and traders are 1.5 feet out the door already.

<comment><USER80> There's also a length limit to submission titles. I don't know what it is, but I suspect the original title exceeds it.

<comment><USER78> > Please limit title to 80 characters [0]

[0]: [https://news.ycombinator.com/submit](https://news.ycombinator.com/submit)

</comment></comment></comment></comment></comment><comment><USER81> "Give me control of a nation's money supply, and I care not who makes its laws." \--Rothschild, 1744

</comment><comment><USER82> Because Warran Buffet uses a computer - but only for bridge playing. But he uses a pocket calculator for his investment decisions.

His philosophy is heavily based on soft facts. Appreciation of management and understanding of the business model. This is not only based on logic and numbers. Therefore a computer does not help.</comment>


13 Simple Words That Could Change Your Life Forever (on accountability) - <AUTHOR> http://johnplaceonline.com/stress-management/13-simple-words-that-could-change-your-life-forever/

<comment><AUTHOR> "The only person I can control is myself. So when I accept responsibility for something, I am giving myself power to change it. Yes, life will victimize me, but I cannot control all the random external forces that may harm me X I can really only control myself. Therefore, the position of accountability is the most powerful position of all because it encourages us to look inside ourselves and make necessary changes."

.."Sometimes, it's better to take responsibility, even if the fault is not your own. Only by taking responsibility for your problems can you take control of your life."

</comment><comment><USER2> Yeah, you _could_ accept responsibility for all the bad things that happen to you, or you could just enjoy being yourself and realize that if someone else has a problem with that, its their failure as a human being :P</comment>