How do I find out what version of Linux is running?

Question: Sometimes your scripts need to behave differently on different Linux's. How can I determine which version of Linux a script is running on?
Comment: I'm sure jldugger wants to find out which distribution family the system is running. It's unlikely a script would be affected by the kernel version unless it depends on some /sys or /proc stuff - and even then it's usually easier to assume based on distribution than on kernel.
Comment: By version, do you mean the kernel version?  What distribution?  The distro version?
Answer: There is no cross-distribution way. However:

Redhat and friends: Test for /etc/redhat-release, check contents
Debian: Test for /etc/debian_version, check contents
Mandriva and friends: Test for /etc/version, check contents
Slackware: Test for /etc/slackware-version, check contents

Etc. Generally speaking, check for /etc/*-release and /etc/*-version.

Edit: Found an old (1+ years) bash script of mine lying around that I must have cobbled together over the years (it has an impressive CVS log going back 6 years.) It might not work properly anymore as-is and I can't be bothered to find installed distros to test against, but it should provide you with a good starting point. It works fine on CentOS, Fedora and Gentoo. gyaresu tested it successfully on Debian Lenny.
#!/bin/bash

get_distribution_type()
{
    local dtype
    # Assume unknown
    dtype="unknown"

    # First test against Fedora / RHEL / CentOS / generic Redhat derivative
    if [ -r /etc/rc.d/init.d/functions ]; then
        source /etc/rc.d/init.d/functions
        [ zz`type -t passed 2>/dev/null` == "zzfunction" ] && dtype="redhat"

    # Then test against SUSE (must be after Redhat,
    # I've seen rc.status on Ubuntu I think? TODO: Recheck that)
    elif [ -r /etc/rc.status ]; then
        source /etc/rc.status
        [ zz`type -t rc_reset 2>/dev/null` == "zzfunction" ] && dtype="suse"

    # Then test against Debian, Ubuntu and friends
    elif [ -r /lib/lsb/init-functions ]; then
        source /lib/lsb/init-functions
        [ zz`type -t log_begin_msg 2>/dev/null` == "zzfunction" ] && dtype="debian"

    # Then test against Gentoo
    elif [ -r /etc/init.d/functions.sh ]; then
        source /etc/init.d/functions.sh
        [ zz`type -t ebegin 2>/dev/null` == "zzfunction" ] && dtype="gentoo"

    # For Slackware we currently just test if /etc/slackware-version exists
    # and isn't empty (TODO: Find a better way :)
    elif [ -s /etc/slackware-version ]; then
        dtype="slackware"
    fi
    echo $dtype
}

Note that this will probably only work correctly in Bash. You could rewrite it for other shells.
That being said, you might want to test for features, not for distributions. I'm not using this anymore simply because it became a maintenance burden. It's easier to rely on cross-distribution tools and solutions.

Conceptually, what it does is, in order:

Pull in a known, "common init script function" type of file. Those are distribution-specific. If it doesn't exist, skip to next distribution check.
Check the existence of a specific, known-to-exist, often-used and unlikely to be renamed function from that core script. We do that using the type Bash builtin. type -t returns function if that symbol is a function. We prepend zz to the output from type -t 2>/dev/null because if the name isn't defined the output string would be empty and we'd get a syntax error about a missing left hand to the == operator. If the name we just checked isn't a function, skip to next distribution check, otherwise we found the distribution type.
Finally, echo the distribution type so the function output can be easily used in a case .. esac block.

Edit in case you're trying to run this as a straight script: This script is supposed to get sourced or included from other scripts. It does not output anything on its own if you run it as-is. To test it, source it and then invoke the function, e.g.:
source /path/to/this/script.sh
get_distribution_type

at the bash prompt.

Edit: Please note that this script does not require root privileges. I urge you not to run it as root. Shouldn't harm anything, but there's no need.

Found a link to a relevant mailing list post in the CVS log. Should be useful in unwrapping init script spaghetti.
Comment: I haven't looked at why but it returns to prompt on Debian Lenny (5.0).
Comment: gyaresu, have you actually invoked the get_distribution_type function? I've edited the post to clarify (see at the bottom.)
Comment: @gyaresu: If the above wasn't the problem, can you please try to replace log_begin_msg in the Debian section with log_warning_msg and retry? Might have gotten the function name wrong. In any case, it should have returned "unknown" if that function wasn't htere, but still.
Comment: @Mihai Doh! Sorry. Didn't read the script properly. Was early, no coffee. My apologise.
gyaresu@debian:~/bin$ source server_version.sh
gyaresu@debian:~/bin$ get_distribution_type
debian
Comment: @gyaresu: Thanks! That's good, should help jldugger to know that it works on Debian as well :)
Answer: Don't try and make assumptions based on the distro as to what you can and cannot do, for that way lies madness (see also "User Agent detection").  Instead, detect whether what it is that you want to do is supported, and how it's done by whatever command or file location you want to use.
For example, if you wanted to install a package, you can detect whether you're on a Debian-like system or a RedHat-like system by checking for the existence of dpkg or rpm (check for dpkg first, because Debian machines can have the rpm command on them...).  Make your decision as to what to do based on that, not just on whether it's a Debian or RedHat system.  That way you'll automatically support any derivative distros that you didn't explicitly program in.  Oh, and if your package requires specific dependencies, then test for those too and let the user know what they're missing.
Another example is fiddling with network interfaces.  Work out what to do based on whether there's an /etc/network/interfaces file or an /etc/sysconfig/network-scripts directory, and go from there.
Yes, it's more work, but unless you want to remake all the mistakes that web developers have made over the past decade or more, you'll do it the smart way right from the start.
Comment: (Expanding on this answer) Feature-detection is preferable in some situations, but make sure you never try and guess the distribution from the features you detect! Don't misread the answer and work out whether the platform is RedHat based on what files are in /etc. If you really do need the distribution name, check lsb_release (or /etc/redhat-release, and so on).
Answer: You can find the kernel version by running uname -a, finding the distro version is dependant on the distro.
On Ubuntu and some other OS' you can run lsb_release -a or read /etc/lsb_release
Debian stores the version in /etc/debian_version
Comment: +1 for lsb_release (also works on Red Hat derivatives if the right package is installed)
Comment: for just the description 'lsb_release -ds'.
Answer: Most distro's have a unique method of determining the particular distribution.
For example:
Redhat (And derivatives): /etc/redhat-release

SUSE: /etc/SUSE-release

There is a standard out there known as the Linux Standard Base or LSB.
It defines that there should be a file called /etc/lsb-release or a program called lsb_release that will echo back information about your linux distro.
lsb_release -a

Comment: And of course, `lsb_release` does not exist on CentOS 6.
Answer: python -c 'import platform ; print platform.dist()[0]'

code:
http://hg.python.org/cpython/file/2.7/Lib/platform.py
Comment: Great idea. I would suggest using ```python -c 'import platform; print(platform.dist()[0])'```, because that way it also works if the normal python defaults to python3.
Answer: In addition to the other answers: If you just want to parse one file, most distros personalize the tty login via /etc/issue e.g.:

Welcome to SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel \r (\l).

And yes I know it's suboptimal. :)
Comment: It may be sub-optimal, but it is in the same place.
Answer: facter is a handy tool for this sort of discovery, although it probably uses some of the methods detailed above, and requires Ruby.
Answer: All you need to do is type uname -a at your favorite shell. That will print out the kernel name and version.
Answer: You can also get the version by
cat /proc/version

o/p:

Linux version 2.6.17-13mdv (email@example.com) (gcc version
  4.1.2 20070302 (prerelease) (4.1.2-1mdv2007.1)) #1 SMP Fri Mar 23 19:03:31 UTC 2007
Answer: I've found that cat /etc/*release* almost always works.
Answer: I concur with Mark, Adam and Mihai (can't vote up due to insufficient reputation).  Solutions based on LSB and its relative FHS will work with most distributions and are likely to continue working in future.  LSB and FHS are your friends.
Answer: The version of linux is a hard question. If we look at it narrowly we have the kernel version you can get with "uname -r". Distribution version is mostly irrellevant. Some distributions are better (enterprise distributions such as Redhat Enterprise Linux). Other distributions like Gentoo are basically moving targets that have no sensible version at all.
If you need to do things based on version, take a look at major components that are relevant to you:
Component       Version command
glibc           /lib/libc.so.6
gcc             gcc --version
X               xdpyinfo
libX11          pkg-config --modversion x11
gtk+            pkg-config --modversion gtk+-2.0
qt-4            pkg-config --modversion QtCore

   etc...

Answer: You can also check the Grub menu, usually gives you a bunch of distro/version info :-)
Answer: FusionInventory is a cross-platform lightweight inventory tool that can get this information on many Linux distros but also on BSDs, Windows, MacOS X and other unices.
If available, they use lsb_release (as mentioned a few times above), but if not they have a very useful list of files and regular expressions to check the distro name and version: https://github.com/fusinv/fusioninventory-agent/blob/2.2.x/lib/FusionInventory/Agent/Task/Inventory/Input/Linux/Distro/NonLSB.pm#L16.
I would recommend using FusionInventory itself to get this information, rather than reimplementing your own scripts with this logic, as their community will maintain this functionality up to date. You could either use the agent on it's own (it outputs a XML/JSON file which is easy to parse) or couple it with a wider solution to manage the machines in your network like GLPI or Rudder, depending on your needs.
Comment: This would be a good solution if it didn't have external dependencies of Perl modules


Contamination concerns when removing stopper?

Question: This might seem dumb to most, but it's only my third batch and I just need some reassurance. So, I occasionally pop out the rubber stopper on my carboy to smell the brew and see if anything funky comes up so I know where to go with my dry hops in a couple weeks. While I was sitting around last night something hit me, when I pull the stopper out, I put it back in without sanitizing. Now it's only out for like 5 seconds tops and I hold it by the airlock, so I don't ever touch the rubber that goes back in the hole, and I don't set it down. Should I worry about contamination? I've only done this 4 or 5 times. It's just smelling AMAZING and I don't want to get through all of this and have caused an infection from my own curiosity.
Answer: As jsled says you have no worries. You are doing the right things, not touching it or putting it down. If just for a few seconds to check on the brew you'll be fine, also you will gain experience regarding how your brew evolves over time.
You should not worry as you are not setting it down for it to pick up bacterial contamination. Yes there is a tiny chance that a little critter will fall from the ceiling in the short interval whilst the stopper is out, but the risk is far outweighed by the learnings you will gain.
Comment: I always give the stopper a quick spray with StarSan before pulling it out and before putting it back in, just in case.
Comment: I agree, but there should be some reasoning behind why we shouldn't worry about this sort of thing.
Comment: @jonpd, the reasoning is the same as that behind the difference between sanitation and sterilization. You're not trying to eliminate every single living thing (sterilizing), you're trying to go about making beer while exposing it to as few microbes (and as little oxygen) as possible within the limits of patience/sanity (sanitizing).
Answer: No, you don't need to worry about contamination based on what you describe.
Answer: You shouldn't be concerned. When you removed the stopper, you didn't allow it to come into contact with a non-sterile (potentially bacteria-ridden) surface, so there was no potential for transfer of bacteria onto it (other than contact with the air, which does not pose a significant risk).
Answer: I have seen a lot of times that there is a layer of protection, when the ferment is going strong, the pressure of the bubbles actually push any contaminants out of the stopper hole.  The layer of foam created by the ferment is also a protection from contaminants.  My airlock had the ferment bubbling into the airlock, so I had to take it off and clean it, and add it back.  Everyone stated that this is natural and if done correctly it would not contaminate the batch.
There is always a possibility of contamination, but if you are aware of it and protect your brew from it, you should be fine.  There are natural combatants to contamination going on when the ferment is in full force.


Could the president push a bailout for industries affected by a disaster without congressional support?

Question: There's a lot of talk lately about bailout packages being offered to airlines and other affected industries.  Assuming that the president was in support of a bailout and was willing to do whatever was (legally) necessary to achieve it, and further assuming congress was strongly against it.  Could the president force said bailout to happen?
Answer: Only Congress can increase or decrease the government's budget. This is enshrined in the US constitution Article I, section 9, clause 7:

No Money shall be drawn from the Treasury, but in Consequence of Appropriations made by Law;

Laws are made by Congress. So if the President wants the United States to take on additional debt or increase taxes in order to fund this bailout, then Congress needs to make a law which says so.
However, since February 15th 2019 there is precedent for using an emergency declaration to reallocate funds Congress already cleared for one government program to another. On that date, the President of the United States used the declaration of a national emergency to move funds earmarked for the Department of Defense to fund border security measures at the US-Mexican border.
However, this measure was moving funds which were already allotted for one government purpose to another. It did not allow the US government to spend more money than it was allowed to according to the budget. So if the US president would use an emergency declaration in order to bail out industries affected by a disaster, and Congress wouldn't cooperate, then the President would have to decide where to take that money from.
Comment: I haven't followed this very closely, but Wikipedia seems to also say that the US Supreme Court, while it has allowed Trump's relocation as temporary measure, has yet to ultimately decide on its legality.
Comment: And the legal battle seems to continue as of March 2020 https://www.salon.com/2020/03/04/19-states-sue-trump-administration-over-plan-to-reallocate-funds-to-southern-border-wall_partner/
Comment: Not all transactions are on budget.
Answer: There are existing statutes that give various bodies in the Executive Branch the authority to loan money without a Congressional appropriation or further act of law. For example, the Federal Reserve, which is not in a direct chain of command from the President, has such authority.
These loans are limited in purpose and amount, but it isn't impossible that bailouts in the form of loans could be accomplished by Executive Order, although actual spending as opposed to loans would have to be supported by some existing appropriation to be made without a new Congressional authorization.
Answer: Nope. He could perhaps redirect already allocated moneys, within the limits of the laws allocating them by congress, but any significant bailout will require congressional approval of new funding.


How to make OS X not remember ssh key passphrase

Question: I have a passphrase on my ssh id_rsa key, and OS X  asks me for the passphrase the first time I want to use it, but even if I decline to add it to my keychain, it doesn't ask me for a passphrase again until I reboot. How can I make it ask me for a passphrase every time I try to use it?
Comment: Have you tried fully closing out your Terminal session?
Comment: yup that doesn't work. only thing that works is logging out as the user altogether. Maybe theres a command i can type into the terminal to tell it to forget the passphrase?
Comment: how do you " decline to add it to my keychain,"? What does `ssh-add -l` prints after the first use? And what about `echo $SSH_AUTH_SOCK`?
Answer: While reading GitHub documentation about adding ssh key to the ssh agent, I thought you might be able to prevent loading keys in the agent by modifying the ~/.ssh/config file like so:
Host *
 AddKeysToAgent no
 UseKeychain no
 IdentityFile ~/.ssh/id_rsa

Answer: Execute following command in Termina.app will disable Keychain integration, thus ssh will always ask you for a passphrase:
defaults write org.openbsd.openssh KeychainIntegration -bool false

Pleases note:

This is undocumented, but as far as I know, it works from macOS 10.6 (Snow Leopard) to 10.11 (El Capitan)
It will affect all private keys, not just id_rsa
Set the value to true if you want to recover the default behavior
Kill ssh-agent process or restart your Mac if it has no effect after executing the command
Comment: its still remembering the passphrase unfortunately, even after doing this. I'm on Sierra now but it was still happening even with El Capitan...
Comment: @MohamedHafez Maybe go into the Keychain app and delete the entry? (just search for ssh)


What should I do with possible frostbite: Swollen feet after putting them in warm water

Question: My feet were frostbite, so I put my feet in hot water when I got home. Ice was forming in my shoe. After I put my feet in the water they puffed up A LOT, they were badly swollen!!! Should I get medical help? I can't bend my feet because they're so badly swollen!
Comment: Yup. Get medical attention asap.
Comment: @BenediktBauer, couldn't this be useful? Your right, if I had this problem, my first reaction would be to contact the doctor not write a question here! But if a good answer is added here, anyone in the future searching for an answer to this may find this information useful. PS, **Get medical attention**.
Comment: Questions asking for medical advice are not really good to answer in this Q&A format, especially when related to problems that would require more or less immediate reaction. You surely should get qualified medical advice as soon as possible.
Comment: @BenediktBauer I agree that when an immediate answer is required, posting a question here doesn't make sense. But as Liam pointed out, answering the question will help others in similar situations. No harm in having the question here. If you still feel unsatisfied, please close the question so that others can air their opinion on the same.
Comment: @Unsung My comment was not intended as "This question is really really bad, we should close it and definitely not answer it" but more like some educational impulse to raise some sensitivity about the problems around medical questions.
Comment: Happened to me (having ice in my shoe). Got lucky to keep my toes. Now I carry **frostbite dressing (good also for burns, light and cheep, very easy to apply)** and gaiter in winter in the mountains.
Answer: Yes, you should seek medical expertise.
From healthcentral.com:

When the skin has thawed and rewarming is complete, cover the damaged
  skin with bandages and warm clothing. Contact your doctor or go to an
  emergency room.
Answer: Just to add to PPL's answer. The UK National Health Service has good practical advice on this also. It's available here. But to summarise some of the relevant points:

Treatment for frostbite depends on how severe your symptoms are. You
  should always seek medical attention if you suspect you or someone
  else has frostbite.
If symptoms are severe, go immediately to your nearest accident and
  emergency department

The affected areas need to be re-warmed. Do not try to do this until
  you are out of the cold. If the warming process is started and the
  frozen parts are re-exposed to the cold, it can cause further
  irreversible damage.
Re-warming should be done under medical supervision because:

it can be a painful process that requires access to painkillers and    expert medical assessment
the best results are achieved using a whirlpool bath that contains a mild antiseptic

Severe frostbite
In cases of severe frostbite, you will need to be admitted to a
  specialist unit that has experience in treating these sorts of
  injuries.
This is often a specialist burns unit, as exposure to very high
  temperatures can cause the same sort of injury as exposure to very
  cold temperatures.


stop android emulator call

Question: I am working on an Android application, having functionality like voicemail.
I am using BroadcastReceiver to get dialing events.
I have to get the event "WHEN CALL IS UNANSWERED (not picked after few rings) FROM RECEIVER".
I will do some actions on caller end against this event.
I am using AVD emulator, and
I do call from one instance to another instance and it calls perfectly,
but the problem is:  It continuously calls until I reject or accept the call.
This way I cannot detect that "CALL IS UNANSWERED AFTER A NUMBER OF RINGS".
So I want the Caller emulator to drop the call after a number of rings (if unanswered) like a normal phone.
I can do it (drop the call after some time) by writing some code, but I need the natural functionality of phone in the emulator.
Can anyone please guide me? Is there any settings in the emulator? Or something else?
The code is shown below in case it helps:
<code>public class MyPhoneReceiver extends BroadcastReceiver {

    @Override
    public void onReceive(Context context, Intent intent)
    {
        Bundle extras = intent.getExtras();
        if (extras != null)
        {
            String state = "my call state = " + extras.getString(TelephonyManager.EXTRA_STATE);
            Log.w("DEBUG", state);
        }

    }
</code>
Comment: http://www.google.com/codesearch/p?hl=en#zvQ8rp58BUs/trunk/phone/src/i4nc4mp/myLock/phone/CallPrompt.java&q=itelephony%20package%3ahttp://mylockforandroid%5C.googlecode%5C.com&d=0
Answer: Hi i think this is impossible with reference to the link:
http://android.bigresource.com/Track/android-sr1t1eagx/
Regards

stop android emulator call


Mac efi bootloader doesn't show ubuntu, and GRUB load automatically

Question: Right now when I start my computer GRUB loads, and then I can select if I want to start either OS X or Ubuntu, and if I start the computer holding down the alt-option key to load Mac's efi bootloader Ubuntu doesn't show up. And therein lies the problem, how can I set things up so when I start the computer it simply boot's into OS X, and when i Load Mac's efi loader I can select to boot Ubuntu, through GRUB? I used to have it set up like this, I don't remember what I did as it was a few years ago. Might be a different way to set things up, if you catch the gist of my problem, I don't use Ubuntu a lot and it is rather cumbersome to load Apple's efi bootloader every time I want to start OS X.
Answer: "efibootmgr is a userspace application used to modify the Intel Extensible Firmware Interface (EFI) Boot Manager. This application can create and destroy boot entries, change the boot order, change the next running boot option, and more."
sudo efibootmgr -v

will return a list of your present boot entries. For example:
BootCurrent: 0000
Timeout: 5 seconds
BootOrder: 0000,0080
Boot0000* ubuntu
Boot0080* Mac OS X

Change order:
sudo efibootmgr -o 0080,0000


Why Goku doesnt use the Super Saiyan Blue kaioken anymore?

Question: In Dragon Ball Super, when fighting Hit, Goku used the super saiyan blue kaioken or also called super saiyajin god super saiyajin kaioken. With this technique, he could get even 10 times stronger. But now he's inferior to Black, and he has been defeated 2 times by him, but he doesnt use it as a last resource. Why is that?
Comment: probably because it was filler/non Canon in the Manga, but who knows. Goku Beat hit by utilizing SSG form, then just before the Time stop, switched to SSGSS, throwing Hit off enough for a decisive victory, though he ofcourse gave up after that like in the anime.
Comment: well now he just used it again
Comment: Ya, in Anime Canon he can use it, though of course it still has its potential costs. The anime is becoming torn between following what the Manga will do, and its own thing, and its causing some pretty big inconsistencies.
Comment: @Ryan To be fair, for _Super_, either the manga is following the anime, or they're both being produced simultaneously (with neither one following the other); I can't remember which.  The anime doesn't actually have filler, because of this, because it doesn't need to give the manga time to get further ahead.  If I remember correctly, Toriyama comes up with the storylines for both, then their creators work from there (explaining the differences between the two).
Answer: There are a ton of disadvantages using Super Saiyan Blue Kaio-ken:

It has only a 10% chance of working and a 90% of failing, which had a chance of killing him.
Goku is only able to maintain Super Saiyan Blue Kaio-ken x10 for short periods of time because it puts a great deal of strain on his body. While Potara fusion could last for an hour.
Goku began to suffer severe muscle convulsions as he powered back down to base form and developed Delayed Onset Ki Disorder which includes significant decrease in ki control and capacity.
King Kai warned Goku to not use this form as the user might incapable of getting stronger.

Though, Goku did used Super Saiyan Blue Kaio-ken once to escape Fusion Zamasu's grasp and send him flying with a swift kick. Goku manages to destroy Fusion Zamasu's Ring of Light, but quickly returns to base form, unable to move. See here for more details.


How do speakers which are silent from the back work?

Question: Trying to work out how speakers which are silent from the rear work -- I.e. they have no bass spill out the back. I have recalled someone saying something about some clever technology, but I'm unable to find it.
Comment: Is this regarding "cardioid" subwoofers?
Comment: I've been trying to find the correct term and failed at that, hence the question. A subwoofer with a cardiod pattern would do (how does that work?) but I've also heard of speakers that cancel bass behind them -- obviously treble isn't an issue since that's quite directional.
Comment: Never heard of, I too wonder if you can give an example? Quite frankly, to my ears it sounds impossible - bass travel like water through everything it can squeeze itself through, but if there actually is a way to direct it, it could be extremely useful in recording-sessions!
Comment: check out http://meyersound.com/products/mseries/m3d-sub/ -- they have a cardiod pattern and are marketed as directional.
Answer: "Cardioid" low end speakers (usually subwoofers in my experience) are usually 2 (or more) drivers in a single enclosure which are intentionally fed signals that are delayed and out of phase with each other in a certain way to cancel out sound in the direction of the back of the cabinet.
This approach sacrifices power for directionality in order to reduce sound bleed to unwanted areas.
You can achieve similar results with individual enclosures with some processing.
Math to follow soon.
Comment: "A pair of 18-inch drivers is mounted at the front of the cabinet, and works in conjunction with a pair of 15-inch drivers facing the rear and is driven by a sophisticated phase manipulation circuit. The resulting directional pattern assures that very low-frequency energy does not spill onto the stage or cause excessive reverberation." - from the product description of the M3Ds linked on the question.  Sounds like you hit the nail on the head. +1
Answer: The M3D sub you mentioned includes a technical description of the function directly on the product page.

A pair of 18-inch drivers is mounted at the front of the cabinet, and
  works in conjunction with a pair of 15-inch drivers facing the rear
  and is driven by a sophisticated phase manipulation circuit. The
  resulting directional pattern assures that very low-frequency energy
  does not spill onto the stage or cause excessive reverberation.

DoritoStyle's answer hit this squarely on the head, but I wanted to offer some more expansion on the concepts.  Basically you put a set of speakers behind the main speakers.  Since you know the distance from the front speaker to the back speaker, you can calculate the delay needed for sound to come out the second speaker such that it is the inverse of the sound just reaching it from the main speaker.
This inversion will effectively cancel out the sound heading directly backwards and destructively interfere with the sound heading out in other directions.  By setting the speaker distance carefully, they can make it so that the amount of destruction heading backwards is not as bad as the destruction going forwards (when the sound from the second speaker reaches the main speaker with a constant frequency being produced.)
It is worth noting that outside of a relatively small frequency band, this technique starts behaving quite erratically as you move through frequencies that don't align as well from to the distance between the main and secondary speakers.  You could probably process it such that it still cancels going backwards, but at certain frequencies, that will also produce varying levels of cancellation going forwards as well.  You could counteract that variation by providing less cancellation at certain frequencies, but then you will get a really weird sound out of the back of the speaker, with some frequencies almost unaltered.
Comment: So the "cardiod" pattern really only applies at a certain set of frequencies?
Comment: Well, you can make the proper sound to play going backwards to cancel out any frequency if you want, but some frequencies will also destructively interfere going forward unless you reduce the level of cancellation.


Brownian motion inside a domain

Question: Consider a regular domain $D \subset \mathbb{R}^d$ and a Brownian motion $B_t$ conditioned to stay inside $D$ for time $t \in [0,T]$. In the limit $T \to \infty$ the conditioned Brownian motion behaves like an homogenous diffusion in $D$ whose invariant distribution is the first eigenfunction of the Laplacian in $D$ (which gives another proof that the first eigenfunction is positive).This should also work for more general diffusion processes. Where can I find a reference for this (and similar) results?
Comment: Stating the obvious here, but it's not going to work for just *any* diffusion process. It won't work for an OU process, it won't work for a non isotropic diffusion process, etc.
Answer: A reference for this might be in the paper of Banuelos "Intrinsic ultracontractivity and eigenfunction estimates for Schrödinger operators"  .  Between equations (1.3) and (1.4) he discussed the semigroup of Brownian motion conditioned to remain forever in D.
Answer: It is hard to say what is a "typical" diffusion process without knowing further details, so let us assume that it is at least governed by a transition kernel, and in particular its solution to be positive a.e. at any initial data if its initial data is positive a.e.: typically, diffusion equations on smooth domains with Dirichlet or Neumann boundary conditions satisfy these properties.
Then, it is known that the semigroup is irreducible (i.e., the motion is not eventually confined in any subdomain) if and only if a strong parabolic maximum principle holds, i.e., the solution is immediately strictly positive a.e. whenever the initial data is merely non-negative and not identically zero. For this reason, such a semigroup can not possibly be confined in any non-trivial subdomain for some time but eventually spread everywhere. Does this answer your question?
Answer: See p.83 in
Engländer, János; Kyprianou, Andreas E. Local extinction versus local exponential growth for spatial branching processes. Ann. Probab. 32 (2004), no. 1A, 78–99.
It is the h-transformed diffusion appearing there.
Answer: A related result can be found in `A Fleming-Viot particle representation of the Dirichlet Laplacian' by Krzysztof Burdzy, Robert Holyst and Peter March.

Brownian motion inside a domain


Migrating pages having custom layout to Office 365

Question: We are migrating SharePoint 2013 to SPO. we are getting error while migrating the pages having custom page layout using Sharegate. With Sharegate layout mapping data is getting lost. So is there any way that we can migrate and retain the data of pages having custom layout? There are some pages where layout is linked to master pages as well.
Answer: As far as I understand, you just want to migrate the data. To lose the custom layout is acceptable.
The SharePoint Migration Assessment Tool (SMAT) assesses pages prior to migration. The migration is then done with the SharePoint Migration Tool.
The SharePoint Migration Tool reverts the custom layouts but migrates the content as statet here:

Customized files are out of the box SharePoint files that have been modified by a user. A common example is using a tool like SharePoint Designer to open a site and modify the default.aspx file of a site. During migration, these pages will be reverted to their uncustomized state.


How to approximate a solution to a matrix equation?

Question: Suppose a matrix equation $Ax = b$ has no solution ($b$ is not in the column space of $A$)
How can I find a vector $x^\prime$ so that $Ax^\prime$ is the closest possible vector to $b$?
Comment: This is studied in every basic linear algebra course.
Comment: I so wish I hadn't made the half-hearted attempt to populate the site with questions in its first week of existence, before the level of appropriate questions had been established. Just delete it. And https://mathoverflow.net/questions/72/whats-an-example-of-a-function-whose-taylor-series-converges-to-the-wrong-thing and https://mathoverflow.net/questions/81/when-does-a-matrix-equation-have-a-solution -- or migrate them to math.stackexchange.com.
Comment: But give me credit for asking three of the first 100 questions, at least. That's got to be worth something.
Comment: @MarkSapir But I am serious -- if you or anyone else can delete or migrate this, please do.
Comment: I did not notice that the question was posted in 2009. I cannot delete a question. I believe you can do that yourself if you want.
Comment: @MarkSapir No, not with an upvoted answer. /shrug.
Answer: I want to correct something in both Anton's ans las3rjock's answers.  I hope that I am merely correcting bad notation.
The way to find x' (using the notation in the question) is, as correctly stated in each of the answers above, to derive the associated problem AT Ax' = AT b and solve that via Gaussian Elimination (also called Gauss-Jordan Elimination, the difference is technical and not important here).  This is guaranteed to have a solution.
The fact that this is the correct solution to the problem relies on the properties of the "closest point" of a point to a subspace.  We want the closest point to b on the subspace Im A.  Call this b'.  The properties of the "closest point" imply that the difference, b - b', is orthogonal to everything in Im A.  It's simple to draw a picture to convince yourself that if c is in Im A and b - c is not orthogonal to everything in Im A then it is possible to "nudge" c a little, either away or towards the origin, to c' so that b - c' is shorter than b - c.
So b - b' is orthogonal to everything in Im A.  Since being orthogonal to something is a linear condition, it is sufficient to check that b - b' is orthogonal to a spanning set for Im A, for which we can take the columns of A.  As we are using the standard inner product, this means that for each column, say a of A, aT(b - b') = 0.  Putting these together, we obtain the relation AT(b - b') = 0.  As b' is in the image of A, there is some x' such that b' = A x', whence we see that x' satisfies ATb - ATA x' = 0, and get the desired formula on rearranging.  This also guarantees the existence of a solution to this equation.
Using the fact that the closest point is the unique point b' in Im A such that b - b' is orthogonal to everything in Im A, we can run this argument backwards to see that if x' is a solution of AT A x' = AT b then Ax' is the closest point to b in Im A.
Where the above answers go wrong is to then talk about the matrix (AT A)-1 AT.  The problem with this is that ATA may not be invertible (take A to be the 2 by 3 zero matrix).  There is a matrix which when ATA is invertible is (ATA)-1AT and this is called the pseudo-inverse.  Essentially, AT misses the kernel of ATA meaning that the composition is always well-defined but it might not be decomposable as the  notation suggests.
This notation may be standard, of that I don't know, but if it is then it is bad notation because it suggests a property that may not hold.  At the least, it should always carry a rider to make clear that the notation is merely suggestive and not to be taken literally.
Comment: +1 Good catch. I wasn't paying attention. I've modified my answer to explain how to reduce to the case where ATA is invertible.
Answer: Edit: Andrew is absolutely correct that using $\left(A^\mathsf TA\right)^{-1}$ is at best sloppy (in my case just a flat-out error) because $A^\mathsf TA$ may not be invertible. If $A^\mathsf TA$ is not invertible, it is because there is a linear dependence among the columns of $A$. In this case, you can remove some of the columns without changing the image until they are linearly independent. The explanation I provide below works once you've removed these "extra" columns.
Anon's and las3rjock's answers are correct: the $x^\prime$ you're looking for is $\left(A^\mathsf TA\right)^{-1}A^\mathsf Tb$. But I wanted to add a neat explanation I've heard for why this produces the correct answer.
The image of $A$, vectors of the form $Ax$, are all linear combination of the columns of $A$ (the entries of $x$ being the coefficients in the linear combination). So if no solution to the equation $Ax=b$ exists, finding the "best approximate solution" amounts to asking for is the projection of $b$ onto the image of $A$.

Claim: The operator $A\left(A^\mathsf TA\right)^{-1}A^\mathsf T$
is the orthogonal projection onto the
image of A.
Proof:

This operator is equal to
its square, so it is a projection:
$$
A\left(A^\mathsf TA\right)^{-1}A^\mathsf T A\left(A^\mathsf TA\right)^{-1}A^\mathsf T =A\left(A^\mathsf TA\right)^{-1}A^\mathsf T.
$$

For any vector $Ax$ in the image of $A$, we have
$$
\left(A\left(A^\mathsf TA\right)^{-1}A^\mathsf T\right)Ax=Ax,
$$
so the image of this projection is equal to the image of $A$.

The projection is orthogonal. Let me denote inner product by $\langle\cdot , \cdot \rangle$. Suppose $y$ is a vector orthogonal to $Ax$ for all $x$, so $\langle y, Ax\rangle =0$ for all $x$, then we need to show that
$$
A\left(A^\mathsf TA\right)^{-1}A^\mathsf T y=0.
$$
For any vector $z$, we have (by the
defining property of transpose)
$$
\langle A\left(A^\mathsf TA\right)^{-1}A^\mathsf T y, z\rangle =\langle y , A\left(B^\mathsf T z\right)\rangle =0
$$
where $B=\left(A^\mathsf TA\right)^{-1}A^\mathsf T$. Since
$A\left(A^\mathsf TA\right)^{-1}A^\mathsf T y $ dots to zero with every z, it must be zero.

So if $b$ is not in the image of $A$, the closest vector that is in the image of $A$ is $b^\prime = A\left(A^\mathsf TA\right)^{-1}A^\mathsf T b$, which is clearly $A$ applied to $x^\prime =\left(A^\mathsf TA\right)^{-1}A^\mathsf T b$.
Comment: Down-voted due to presence of (ATA)-1.
Answer: Andrew has a correct answer in the pseudoinverse $A^+$, which is characterized by the property that $x = A^+b$ is the shortest vector that solves $A^TAx = A^Tb$ (equivalently, $x$ has zero nullspace(A) component).  Computationally, it is typically found using a singular value decomposition: if
$\displaystyle A = U \Sigma V^T = \left[ \begin{array}{cc} U_{col} & U_{null} \end{array} \right] \left[ \begin{array}{cc} \Sigma_{pos} & 0 \\\ 0 & 0 \end{array} \right] \left[ \begin{array}{cc} V_{row} & V_{null} \end{array} \right]^T,$
then the pseudoinverse is $A^+ = V_{row} \Sigma_{pos}^{-1} U_{col}^T$.
I'd like to mention that the pseudoinverse is both rather computationally expensive and unstable in the presence of noise.  In particular, if $b$ is given by taking real-world data with limited accuracy, and $A$ is ill-conditioned (e.g., singular), the output of least-squares can vary wildly with the error.  One common approach to rectify this is Tikhonov regularization, which typically means minimizing $\Vert Ax - b \Vert^2 + \alpha \Vert x \Vert^2$ for some small $\alpha$.  This generically yields a nonsingular optimization problem, which can be computed quickly by Gaussian elimination, and as $\alpha$ approaches zero, the solution approaches the pseudoinverse solution.  It will not in general yield an exact solution, but there are error-minimizing heuristics (e.g., using the Discrepancy Principle) for choosing $\alpha$ based on knowledge about the size the noise.
Reference: Strang, Computational Science and Engineering
Answer: As Anon mentions, linear least squares is the standard method for solving this problem.  It involves solving the system of linear equations
$$
A^\mathsf TA x = A^\mathsf T b
$$
which are known as the normal equations.  If the system is small enough to solve by hand, one can apply Gaussian elimination or calculate the Moore-Penrose pseudoinverse $\left(A^\mathsf TA\right)^{-1}A^\mathsf T $ (assuming $A^\mathsf TA$ is invertible), but the standard computer algorithms for solving linear least squares problems use either Cholesky factorization, QR factorization, or singular value decomposition.
Comment: Down-voted due to presence of (ATA)-1
Comment: I mentioned the Moore-Penrose pseudoinverse because I was describing the situation (small m,n) where it should be fairly easy to determine whether or not (A^T A) is invertible.  I agree that the notation is sloppy, but it's fairly common in science and engineering literature (which tends to be sloppy about assumptions and notation).
Answer: Multiply by $A^\mathsf T$ (transpose) on both sides and solve using Gauss-Jordan elimination. This gives least squares solution.
Answer: Worth a very minor clarification: in the case when $ A^\mbox{T} A$ is not invertible, there is still a single point $y$ in the image (column space) of $A$ that is closest to $b.$ In the annoying extreme that $A$ is the matrix with all entries 0 we will be stuck with $y = 0.$ It is just that when  $ A^\mbox{T} A$ is not invertible  we can no longer guarantee a single ``best''  $ x = \hat{x},$ that is there will be an infinite collection of $x$ satisfying $ A x = y$ with $y$ as above.  Again, when  $A$ is the matrix with all entries 0 then all possible values of $x$ give a "right" answer.
You might enjoy  pages 7-11 and 42-49 in "Kalman Filtering: Theory and Practice" by Mohinder S. Grewal and Angus P. Andrews. The system under consideration is called observable if there is a unique "best answer" $\hat{x},$ which is exactly what happens here when $ A^\mbox{T} A$ is invertible


C++ vs Fortran for HPC

Question: In my computational science PhD program, we are working almost exclusively in C++ and Fortran. It seems like some professors prefer one over the other. I am wondering which one is 'better' or if one is better than the other in a certain circumstance.
Comment: A mix of a high and low level language is better than exclusively using either, in my opinion. E.g. I use Python + C++.
Comment: The answers to this question will be almost purely subjective and thus I'm not sure this question is appropriate.
Answer: As so often, the choice depends on (1) the problem you are trying to solve, (2) the skills you have, and (3) the people you work with (unless it's a solo project). I'll leave (3) aside for the moment because it depends on everyone's individual situation.
Problem dependence: Fortran excels at array processing. If your problem can be described in terms of simple data structures and in particular arrays, Fortran is well adapted. Fortran programmers end up using arrays even in non-obvious cases (e.g. for representing graphs). C++ is better suited for complex and highly dynamic data structures.
Skill dependence: it takes a lot more programming experience to write good C++ programs than to write good Fortran programs. If you start out with little programming experience and only have so much time to learn that aspect of your job, you probably get a better return on investment learning Fortran than learning C++. Assuming, of course, that your problem is suited to Fortran.
However, there's more to programming than just Fortran and C++. I'd recommend to anyone going into computational science to start with a dynamic high-level language such as Python. Always remember that your time is more valuable than CPU time!
Comment: "Always remember that your time is more valuble than CPU time!" As someone who works in scientific research, I couldn't agree more with that part.
Comment: "Always remember that your time is more valuable than CPU time!" As someone who works in HPC I disagree with that part; everything else is spot on.
Comment: "Always remember that your time is more valuable than CPU time!", I would rather code for a month and run in a week! - more can be done once the code is written and others will find the code you write more useful too.
Comment: "Always remember that your time is more valuable than CPU time!" - I'd like to throw in my 2 cents - using several hundred nodes, each with 10+ cores to run some program for several weeks can be construed as a horrible waste of a most precious resource, if a couple more weeks could produce a code that runs in only a couple of days. Those HPC clusters are a rare and expensive common resource.
Comment: Always remember that whether your time or the CPU time is more valuable is highly context-dependent!
Comment: "Always remember that your time is more valuble than CPU time!", code for a week but run for a month, this's quite usual sir!
Comment: Honestly, this is why I love Julia so much. Sometimes it's not as fast as perfectly optimized Fortran, but the situations where the 10-100% difference between Julia and Fortran matters are rare. On the other hand, a 10x difference like you'll get out of Python will often matter a *lot*.
Answer: I think that both C++ and Fortran are good enough and work well.
However I think that Fortran is better for numeric scientific computing, for algorithms that can be expressed using arrays and don't need other sophisticated data structures, so in fields like finite differences/elements, PDE solvers, electronic structure calculations. Fortran is a domain specific language. In particular I think that it is easier to write fast programs in Fortran than in C++, by a scientist (not necessarily a computer science expert).
C++ is a general purpose language, so one can express any algorithm in it, and it is most definitely better for algorithms that can't be expressed using arrays, from HPC field probably some graphs, mesh generators, symbolic manipulation and so on.
It is also possible to write array algorithms in C++, but in my experience, it requires much more computer science knowledge and in general more work (i.e. one needs to create or reuse classes for array manipulation, and handle memory management by hand or using some library like Teuchos from Trilinos). Non-experts tend to write pretty good Fortran programs, but horrible C++ programs (talking from my own experience).
Disclaimer: I personally like Fortran a lot and I prefer it over C++ for numeric computing. I have spent over 2 years of programming in C++ daily, and almost a year programming in modern Fortran daily (in finite elements area). I use Python and Cython a lot too.
Comment: I couldn't disagree with this response more. Our finite element code would not have been possible to write in Fortran. In fact, it started 15 years ago as a mix of plain C and Fortran (the latter being for the numerically intensive parts of the method), and it gradually moved to pure C and then to C++ over the course of several years. The code got consistently shorter, faster, and easier to understand, and it was more capable after each iteration.

I agree with others that point out that C++ gives you plenty of rope to shoot yourself with. Pick the language you're most comfortable with.
Comment: @WolfgangBangerth, see for example Phaml (http://math.nist.gov/phaml/) for a Fortran implementation of pretty much everything that you mentioned.
Comment: Bill, did you use modern Fortran (90 and later additions?). This is very important (I should have been more explicit in my answer about this). Of course, "20.000 lines of Fortran", or f77 usually is not better than well written C++.
Comment: One up for the first answer being balanced. I think C++ and Fortran are by far not the only possibilities in contemporary HPC.

I think it is good to know the strength and weak spots when you decide for Fortran, C++ or Python (or whatever you like). I have seen 20.000 lines of Fortran in one single file, organicly grown over some decades. I personally would not use for anything other than isolated heavy array computing. Not even for anything related to output. So far for a biased comment.
Comment: @OndřejČertík: I think that if you believe that modern finite element programs use "simple" data structures, then you haven't looked at any of them recently. Try implementing adaptive finite elements, hp methods, or multigrid on unstructured meshes using simple data structures. Bill is spot on and I believe I can speak for him in saying that using "modern Fortran" would have made hardly more than a small difference.
Comment: This is a very general question, so I am answering also in general terms. I am aware that other people will have different opinions.
Answer: I'm also throwing my two cents in kind of late, but I've only just seen this thread and I feel that, for posterity, there are a few points that desperately need to be made.
Note in the following that I will talk about C and not C++. Why? Well, otherwise it's apples and oranges to compare a full-fledged dynamically typed object-oriented language with something as static as Fortran. Yes, some modern implementations of the latest Fortran standards can do more than just that, but very few people actually use them, and so when we speak of Fortran, we think simple, static, and imperative language. That's where C is too, so I'll replace C with C++ for the following.
First of all, any discussion of Fortran/C having better compilers is moot. Dedicated C/Fortran compilers are a thing of the past. Both gcc/gfortran and icc/ifc are just different front-ends to the same back-end, i.e. your program will be transformed into an abstract description by the front-end and then optimized and assembled by the back-end. If you write, semantically, the same code in Fortran or in C, the compiler will, in both cases, produce the same assembly which will run just as fast.
This now leads to my second point: why do we still see differences? The problem is that most comparisons are made by Fortran programmers trying something in C or vice-versa. Ever notice how most authors or poets prefer to write in their native languages? Would you want to write poetry in a language in which you don't feel completely confident or at home? Of course not... I myself consider C to be my "native" programming language. I did, however, also spend three years working in a group that used only Fortran, in which I have achieved a certain level of fluency. I would, however, never write anything on my own in Fortran since I'm more comfortable with C and, as a consequence, the resulting code will be better, whatever you define that as.
So the main difference is in the programmer, not the language. So there are no differences? Well, not quite. Here are a few examples:

SIMD: Whether it's SSE, SSE3 or AltiVec, if you want to use them in Fortran, you better hope and pray that the compiler guesses exactly what you want and does it so. Good luck. In C you generally have intrinsic functions for each architecture, or, more recently, general SIMD vector types in gcc. Most Fortran compilers will only use SIMD instructions to unroll loops, but if you have a kernel which works on short vectors of data in a non-obvious way, the compiler will very probably not see it.
Different hardware architectures: The whole CUDA architecture is built around kernels in C. Yes, the Portland Group now has a CUDA-capable fortran compiler too, but it's commercial, and most importantly, it's not from NVIDIA. Same goes for OpenCL, for which the best I could find is a recent project which only supports a few basic calls.
Parallel programming: Yes, both MPI and OpenMP work just fine with both C and Fortran. However, if you want real control of your threads, i.e. if you have a fully dynamic shared-memory computation, you'll be out in the cold with Fortran. In C you have the standard pthreads which, while not warm and fuzzy, will still get you through the storm. In general, most computations that rely on access to the operating system, e.g. threads, processes, file system, etc... are better served with C. Oh, and don't try to do your own networking with Fortran.
Ease of use: Fortran is closer to Matlab than C is. Once you've gotten over all the different keywords and how to declare variables, the rest of the code looks like Matlab, making it more accessible to users with limited programming experience.
Interoperability: When you create a struct in C, the layout of the actual data is straight-forward and deterministic. In Fortran, if you use pointer arrays or structured data, the actual layout of the data is strongly compiler-dependent, not straight-forward, and usually completely undocumented. You can call C from Fortran and vice-versa, but don't start thinking it may be as easy to pass anything more than a static array from one to the other and back.

This is all somewhat geeky, low-level stuff, but this is High-Performance Computing we're talking about, right? If you're not interested in how to best exploit the underlying hardware paradigms, i.e. implementing and/or developing algorithms which are best for shared/distributed memory, threads, SIMD vectorisation, GPUs using SIMT, and so-on, then you're just doing math on a computer.
This has gotten much longer that anything I entended, so here's a summary -- a set of take home messages of sorts:

You will write the best code you can in the language which you know best.
There is no difference in the quality of code produced by two compilers which use the same back-end -- it is us who write bad code in one language or another.
Despite feeling more low-level, Fortran is quite a high-level abstraction and won't let you access certain hardware/OS features directly, e.g. SIMD, threads, networking, etc...
Comment: Good response. I don't think however your final comment is necessarily true. I'm a C programmer myself, but you get access to low level things in Fortran through good programming practices. The ideal way to utilize things like SIMD ops is to write code which strongly suggests it (blocking loops out, for example) and let the compiler do it for you. For threading, simply use openMP (pthreads is also usable with some extra work).

Fortran has all of the things you mention it doesn't, just at a level that matters to its typical user: numerical.
Comment: @Reid.Atcheson: Well, if you block-out everything such that the compiler will catch it, then it will work automagically both in C and in Fortran. The problem is, though, how far do you want to trust your compiler? And why do you want to have to trust it in cases when you know *exactly* what you want done? OpenMP does threading, yes, but block-wise. You can trick it into getting different thread pools to do different things, but that is just mis-using OpenMP. Pthreads for Fortran are just wrappers to the C functions. I agree, though, that Fortran is easier if you're not into the details.
Comment: Sure you aren't going to get full-blown 99% peak efficiency relying on the compiler, but you can easily come quite close. Beyond this you either have to use intrinsics or inline ASM. You have to make concessions somewhere for overall programmer efficiency, that's why programming languages exist in the first place.

At the stage that you actually are insane enough to get into the details of intrinsics or ASM (I have been a few times), Fortran isn't a crutch. You'd know how to link in your assembled hand-optimized code anyways.
Comment: @Pedro, Brilliant post. Absolutely brilliant. Thanks a lot for posting. Just found it while randomly rummaging through interesting threads.
Comment: @Reid.Atcheson: Well, I'd argue that for parallel HPC applications, you may well end up far below 99% peak efficiency... And the gcc vector types make using intrinsics a non-issue :)
Comment: Thanks for mentioning that actually. I don't know how the vector types slipped under my radar, I'm going to start using them though.

You will end up far below 99% peak efficiency, I was kind of exaggerating. I'll happily sacrifice runtime for my programming time if I think it'll keep me from pulling my hair out. Still do my best for efficiency. I only resort to a serious optimization attempt if it's absolutely necessary.
Comment: @Pedro, Isn't vectorization/SIMD a bit tougher to do automatically(i.e. compiler) in C than it is to do it in fortran, and, thus, motivating the 'resctrict' keyword on C99?
Answer: From my 15 years of thinking about scientific software: If your code runs 25% faster because you write it in Fortran, but it takes you 4 times as long to write it (no STL, difficulty implementing complex data structures, etc), then Fortran only wins if you spend a significant fraction of your day twiddling thumbs and waiting for your computations to finish. Given that for almost all of us the most valuable thing is our own time, the conclusion is this: use the language that allows you to develop, debug and test your code the fastest, within reason ignoring that it may be slower than maybe possible if you wrote it in Fortran.
Comment: Or maybe use a language that has a decent ecosystem.
Answer: My approach has been to use C++ for everything but computational kernels, which are usually best written in assembly; this buys you all of the performance of the traditional HPC approach but allows you to simplify the interface, e.g., by overloading computational kernels like SGEMM/DGEMM/CGEMM/ZGEMM into a single routine, say Gemm. Clearly the abstraction level can be raised much higher by avoiding raw pointers and switching to opaque classes, but it is a nice first step.
I find the largest downside of C++ to overwhelmingly be the increase in compilation time, but, in my experience, the savings in development time more than make up for it. Another downside is that vendor C++ compilers tend to have more bugs than vendor C and Fortran compilers. In the past year, I think I have run into nearly ten bugs in C++ compilers.
With all of that said, I think that the undoing of scientific packages written in low-level languages (and Fortran) is the reluctance to expose convenient interfaces for sophisticated data structures: most people are satisfied with the Fortran BLAS interface, as it only requires pointers and leading dimensions to describe matrices, but few people would argue that the usual 40-integer Fortran sparse-direct solver interface is anything close to convenient (cf. UHM, SuperLU, PETSc, and Trilinos).
In summary, I argue for using assembly for low-level computational kernels, but higher level languages for everything else, especially when operating on non-trivial data structures.
Note that this post resulted in this comparison of the performance of C and Fortran on the kernel $y := \alpha x + y$.
Comment: Why wouldn't you trust a standard C compiler with appropriate optimization enabled for the purpose of compiling small kernels?  At that level of code size and complexity the difference in what a compiler could pull out of it is unclear.
Comment: I have talked with several people who have told me that, even with appropriate restrict usage, their Fortran was still faster than their C and/or C++ code for some operation like an explicit matrix transpose. I'm not saying that it's impossible to make the C or C++ code as fast, but that the Fortran compiler tends to do a better job.
Comment: Just to give a bit of perspective on the often-repeated, rarely-validated claim that Fortran is faster on numerical kernels: A while back we noticed that the sparse matrix-vector multiply in Trilinos' Epetra package was 30% slower than the one in deal.II. The former was written in straight forward Fortran 77, the latter in straight forward C without the use of 'restrict'. Both had around 10-15 lines of code. Today, Trilinos uses the piece of code hoisted from deal.II.

I'm sure one can find lots of cases where F77 is faster than C. The point is that isn't universally so any more today.
Comment: I have the same experience with the "restrict" keyword (my simple Fortran code was always a little faster). But my expertise is limited, and I simply don't have time to invest into understanding the generated assembly from gcc. So I simply use Fortran, it's simple and it's fast.
Comment: @JackPoulson: The compiler argument is something I hear quite a bit from the Fortran community. Unfortunately, most compilers, e.g. gcc or ifc/icc, use different language front-ends for the same back-end. The machinery doing the optimization and code generation is identical and therefore the differences in the results are most probably due to differences in the familiarity of the programmer with the underlying language...
Comment: @Pedro: I have made your exact argument for years, but, as I mentioned, even with proper restrict usage there tends to still be a small difference in performance. I would expect the situation to change when vector intrinsics are introduced, but that is another story.
Comment: @JackPoulson: Do you have a concrete example of such a code?
Comment: @Pedro: It was a kernel to explicitly transpose a matrix that a colleague wrote. After I fought for him to use 'restrict' he claimed that his F90 code was still faster. I do not have a copy, but I will reiterate that I am referring to run-of-the-mill code, not highly optimized code. I am sure the performance is equal once you get down to using vector intrinsics.
Comment: @JackPoulson: We're drifting somewhat off-topic here, but if the backend of the Fortran compiler was clever enough to *automatically* "vectorize" any loops, the backend of the C compiler would do the same, since it's the same backend. The problem with *explicit* vectorization is actually the opposite: you can do it in C, but not in Fortran (missing intrinsics).
Comment: @Pedro: Please prove me wrong! I do not like my conclusion, but I am speaking from my (somewhat limited) empirical data. I do not dispute your analysis.
Comment: @JackPoulson: Ok, I've taken up the challenge and formulated it as a question of its own :)
Comment: @WolfgangBangerth, Pedro: The funny thing is that I agree with both of you; the point of my post was to rile people to actually provide hard numbers. I consider my troll wildly successful.
Comment: @JackPoulson: Given the (extensive) follow-up question this comment generated, you might want to edit your answer to point to the (awesome) C vs. Fortran question.
Answer: The problem with C++ is that you have numerous chances to ruin performance, for instance by blindly using STL, exceptions, classes (virtual overhead plus alignment problems), operator overloading (redundant new/deletes) or templates (never-ending compilation and cryptic errors seem benign, but you can waste hours this way).
However, more you gain better access to general libraries and possibly grater visibility of your code (although this strongly depends on the field, and you still have pure C). And you can still compensate the Fortran's lack of flexibility by wrapping its code in a script language like R, Lush, Matlab/Scilab or even Python, Ruby or Lua.
Comment: I think both mbq's and Martin's points are unfair. Yes, there are ways to shoot yourself in the foot if you try to implement a numeric vector for linear algebra purposes using std::list. But that's a silly argument: at least C++ *has* a linked list class that you can use, while Fortran doesn't. It's like saying "Cars drive at such high speed that you could crash into a wall and get injured; you should use horse-drawn carriages instead." It's just a silly idea to trash a high-level language that also supports low-level stuff (e.g. C++) for *having* high-level features.
Comment: I appreciate your opinion, but I do maintain that Fortran isn't as evolved as C++ is :-)
Comment: It is generally a bad idea to apply low-level techniques in high-level languages. For example, the STL is designed to operate on a very abstract level. One has to be aware what the interface is designed for, use it for this task and then get out of the compilers way.
Comment: @WolfgangBangerth No, now you are hurting Fortran -- it is as "low-level" as bacteria are "less evolved" then humans. If you want a car analogy, it should be more like "you can use both Jeep and Lexus to cross a swampy byway, but using the first one hurts less".
Answer: Since I am new here, I was looking through old questions and found this one. Hopefully it's not taboo to answer old ones!
Since no one else has mentioned this, figured I would. Fortran 2003 is almost fully supported by most of the major compilers (intel, ibm, cray, NAG, PCG) even gcc with the (soon-to-be) newest release 4.7. Fortran 2003 (and 2008) is an object oriented language, albeit a bit more verbose than C++. One of things that I think is nice about Fortran is the fact that the standard committee see's scientific computing as it's primary audience (I thank Damian Rouson for pointing this out to me the other day).
I bring this all up not so that C++ programmers become Fortran programmers, but so that Fortran people know that they have more options now besides switching to C++ or emulating object oriented concepts in Fortran 90/95.
One caveat I will add is that there is a cost to being on the bleeding edge of what's implemented in the compilers. If you undertake a major project in Fortran 2003 right now you will stumble across bugs and continually need to update your compiler (especially if you use gcc), though this has gotten significantly better in the past few months!
Answer: Three facts:

F77-style n-dimensional arrays in C: No problem using CnD (a shameless plug, admittedly)
F90's module system is poorly designed and hostile to build environments. (A module's name doesn't have to match its filename, e.g.)
Fortran does not support refactoring well. Pulling some bit of functionality out of a function requires you to touch four places: Actual code, variable declarations, argument declarations, and argument list. C gets by with two places to touch. This compounds the effect of the failure to manage data well (described below): Since small-scale modularity is so painful, just about everybody writes gigantic subroutines.

One personal impression:

Fortran does not work well for managing data. Try returning a pointer to a user-opaque data structure in F77 or F90. (<code>transfer()</code>, here we come)
Comment: "A module's name doesn't have to match its filename, e.g" You must be joking, you can have many modules in one file. Some of them spanning only a couple of lines. They are much easier to create if you do not have to create a file for each of them.
Comment: Refactoring modern fortran is trivial with proper IDEs, like Photran in eclipse.
Comment: Just wanted to add to what @user389 said that, while Photran is great and is the only Fortran IDE that allows refactorings, its parser fails all the time. On the other hand, there's no need to comment on the fact that Eclipse is memory-hungry.
Comment: Hi Andreas! CnD is interesting, I didn't know about it. Ah, you wrote it. :) (f90 also supports slicing, allocatable for arrays and most importantly -- array syntax for multiplication, addition and so on.) I use CMake with Fortran and it works great with modules. What exactly is "argument list"? I don't think I use these, so only 3 places are needed to modify. In C, you typically need to modify the actual code, parameters and a header file, so it's also 3 places (most definitely in C++). Yes, transfer() isn't super nice, but typically you don't need it in practice.
Answer: Fortran is optimized for array/matrix computations and is a thorough pain to work with for any type of text parsing. C and C++ may not match up with Fortran in numerical computing (it's close),but I find it much easier to process text and organize data (i.e. custom data structures) with C/C++.
As others have mentioned, don't count out dynamic interpreted languages (Python et al). They may not offer the face-melting speed of Fortan up front, but they allow you to focus more on solving your computational problem than all the details of implementation. Often you can implement a solution in Python, and if the performance is unacceptable, do some profiling, identify the problem areas, and either optimize that code using Cython or re-implement the entire program in a compiled language. Once you have the problem-solving logic fleshed out, the rest is just implementation and, with a good understanding of computing fundamentals, should be straightforward to represent in any variety of programming languages.
Comment: That's right. For text parsing I also use Python.
Comment: You can also implement part of a Python script in a compiled language e.g. C++ and hook it in. E.g. Boost Python, Swig etc.
Answer: I'm currently working at one of the national labs. Most of the folks around me are mechanical engineers. Chatting with some of the folks in the HPC groups, they're doing mostly Linux and mostly C++. The group I'm currently in does mostly desktop applications and we use Windows and in descending order: C#, FORTRAN, Python, VBA and VB (6, not .NET). Some of the simulation engines we use were written at other national labs in FORTRAN.
Answer: Sorry for digging up an old thread but it seems that even in 2015, Fortran is being used a lot.
I just came across this (alternate link) list which basically is a list of 13 codes approved by DOE's OCLF facility to run on the 300-petaFLOPS Summit machine which will be made available to researchers in 2018. I tried to find the main language used for the code (based on a quick google search) and here is what I found:
<code>XGC Fortran

SPECFEM Fortran

ACME Fortran (Bunch of climate codes)

DIRAC Fortran (Mostly)

FLASH Fortran

GTC Fortran

HACC C/C++

LS-DALTON Fortran (some C)

NAMD C/C++

NUCCOR Fortran

NWCHEM Fortran

QMCPACK C++

RAPTOR Fortran
</code>
So out of 13 codes, at least 10 (based on my quick search) appear to be written in Fortran. Not bad for a 50 year old language.
NOTE: I am well aware that language comparisons are useless but given the number of folks (specially C++ users) that bad-mouth Fortran, I thought it might be worthwhile to mention it.
Comment: I disagree, because my experience at the national labs has, if anything, been the opposite. Most of the new projects I see at Lawrence Livermore are written in C++, and most of the new (or actively maintained) state-of-the-art open-source libraries in ODE solvers, FEM discretizations, and general-purpose scientific computing libraries seem to be in C or C++. Fortran seems to be mainly used in projects that use existing/legacy libraries; I don't see a lot of big, new projects using Fortran, independent of what I think about the language.
Comment: @blochwave As you can read in the link, the projects are for a new machine (with accelerators etc.) that will be online in 2018. So its not like they take a 25 year code and compile it, hoping to run with at good performance. I am quite sure large parts of the codes in the above list have or are been rewritten, as in new code. A number of "new" climate codes are in Fortran as well and used by many agencies in a number of countries.
Comment: Some density functional theory codes also written in Fortran include [VASP](https://www.vasp.at/) and [CASTEP](http://www.castep.org), though as @GeoffOxberry points out, **new** projects do perhaps tend towards C++.
Answer: What Jack P. I think is trying to say is that you should mix and match.  A good piece of software is carefully layered.  Different layers may map more naturally, or efficiently, to different languages.  You should choose the most appropriate language for each layer.  You should also understand how languages can interoperate, which may affect what language you choose for what layer.
A better question is what examples of excellently designed software are out there that are worth studying to learn about how to design layered software.
Answer: Well here is my opinion. To answer this question one should be aware of what scientific computation is. Most of the time it is the numerical solution of Partial Differential Equations (PDEs) on Complex Domains.  Decades ago, PDEs could only be solved numerically on regular domains for e.g. rectangles. These types of domains can be represented by something called a structured grid. Variables on structured grids are most conveniently represented by an array (1,2,3-dimensional). Hence, what is required is efficient and powerful array manipulation capability. And this includes doing vector (array) arithmetic. Hence, FORTRAN was invented. Later, it became essential to solve PDEs in complex domains for e.g. determining stresses in an airplane wing. These complex domains are most conveniently represented by unstructured grids involving for e.g. triangular patches in 2D or hexagonal solids in 3D. And so the Finite Element and the Finite Volume Methods were invented. Variables on unstructured grids are NOT conveniently represented by arrays. You need more complex data structures. Hence, the need for programing languages that can manipulate complex data structures efficiently and with power. And that is where languages like C & C++ comes. These days most of the solution algorithms for solving PDEs use unstructured grids and FORTRAN is not the most natural language for these types of problems. Having said that, there are still areas in which researchers are using structured grids and hence they are using FORTRAN for e.g. in reservoir simulation.

C++ vs Fortran for HPC


İrreversible dopamine antagonist vs. Dopamine agonist

Question: Can a dopamine agonist reverse the effects of an irreversible dopamine antagonist?
Comment: Can you clarify whether you're talking about an inverse agonist or an irreversible antagonist? Your question title says inverse antagonist, which isn't a typical term. The body says irreversible antagonist.
Comment: What do you mean by reverse?  Do you mean withdrawal?  The brain is altered by medications which would do not always understand.
Comment: İ mean unblock.
Comment: William so they both block dopamine receptots then ?
Answer: No. Irreversible antagonism is, by definition, inhibition that cannot be reversed by the agonist. Let me summarize the basics. You can read about this in Goodman and Gillman's The Pharmacological Basis of Therapeutics, Chapter 3, but there are many basic biochemistry textbooks that will cover the same ground.
When an interaction between a ligand and a receptor is said to be reversible, it means that it can associate to form a ligand receptor complex, and then dissociate, leaving the system with free ligand and free receptor. In the most general case, when the ligand is bound, it changes the conformation and activity of the receptor, and when it dissociates, the receptor returns to its original conformation and activity. The effect of the ligand in the biological system is related to the proportion of bound receptors.
Irreversible ligands typically involve covalent bonding (e.g., aspirin, alkylating agents). When the ligand interacts with the receptor, it irreversibly alters the conformation and function of the receptor. Cell or tissue function is only returned to the free receptor state when the particular receptors that have had a ligand receptor interaction have been degraded and replaced by a new receptor.
One of the ways to test whether an interaction is reversible is to try to reverse it. Let's look at how you would do this.
Start by looking at the effect when an agonist binds. The agonist is reversible. When you add it to an assay that contains the receptor, it associates and dissociates from the receptor, but at any moment, some proportion of receptors is bound to agonist. This binding produces an effect you can measure. The effect is related to the overall proportion of receptors bound to agonist at any given time, which is related to the concentration of the agonist. When you increase the concentration of an agonist, more agonist binds and you can measure a greater effect. In this figure from Goodman & Gillman Chapter 3, you can see the log concentration of the agonist on the x axis and the effect measured in an assay on the y axis:

Now consider the effect of an agonist in the presence of an antagonist. Classically, a competitive reversible antagonist binds the same site as an agonist. When the antagonist is bound, it doesn't allow the agonist to bind. So run the same experiment as you did above (by increasing the concentration of the agonist and measuring the effect), but start with a solution that contains some fixed concentration of antagonist. Because an antagonist is competing for those same binding sites, there will be fewer receptors available for agonist binding, fewer receptor/agonist complexes, and a smaller effect.
If the antagonist is reversible, though, you can still get the same effect you got in the first experiment. You just have to increase the concentration of the agonist until it binds the same proportion of receptors. Since both agonist and antagonist are constantly associating and dissociating with the receptor, increasing the concentration of one of them will out compete the other.  In this figure (same source), you can see that in the presence of an antagonist (here, i, for inhibitor), it requires more agonist to get the same effect, but the maximum effect is still the same.

If the antagonist is irreversible you can't get the same effect you got in the first experiment, even by increasing the concentration of the agonist. The receptors have been permanently altered by the presence of the antagonist, and whatever portion of them had some interaction with the antagonist will either not bind the agonist, or will not produce an effect when bound. What you see in the assay here is that increasing the agonist concentration in the assay may increase the measured response, but you are unable to reach the same maximum response. Note here, the figure describes a pseudo-irreversible inhibitor (antagonist). This is just a term for an irreversible inhibitor that doesn't covalently bind the receptor, but has a strong enough affinity for the binding site that, based on the measurements, it looks like it covalently binds. Effectively, the inhibitor is going to sit there on the binding site until the receptor is degraded and will not be outcompeted by the agonist.
Comment: @user57928 You're welcome! Your other question about how long it takes for receptors to degrade, [asked here](https://biology.stackexchange.com/questions/76622/how-long-does-it-take-for-a-blocked-dopamine-receptor-to-be-broken-down-by-the-b) is a lot more complicated than it may seem. Try editing it, though, as suggested by my comment there, and we'll see if we can't get some answers.
Comment: Thank you. Do you know how long it would take for receptors blocked by a irreversible dopamine antagonist to degrade?
Comment: @user57928 As far as this question is concerned, take a look at the help page for [someone answers](https://biology.stackexchange.com/help/someone-answers). If  this answered your question about irreversible antagonists, you should mark it the answer by clicking the gray check mark next to it. Once you have the upvote privilege, you should upvote useful answers as well.
Comment: But in practice I have yet to find any irreversible dopamine antagonists on any commonly prescribed drugs today(if Wikipedia is a valid source).
Comment: There's Haldol off course.


Can a problem with heater stack liner cause outside chimney mortar to crack and fall out

Question: I had "high-efficiency" furnace installed in 2009. In 2011 chimney required full grind-out and repoint job, and chimney saver solution was applied to prevent water damage. Service lasted only 18 months. Company now informs me that they missed identifying a faulty furnace liner, as mortar is coming away from chimney, although there are cracks throughout including front of the chimney alongside the fireplace stack. Is this likely, or is it possible that previous pointing work did not cure because of rain following day? They state that the new furnace is creating moisture and liner is in need of replacement.
Comment: What kind of fuel does the furnace burn?
Comment: It is a gas furnace, Trane XR80 high efficiency 100,000 BTU
Comment: Also, had installed Trane power humidifier at same time
Answer: Rain a day after pointing work will help, not harm, the cure. Freezing weather would harm the cure.
High efficiency furnaces output low-temperature flue gas - more heat in your house, less up the chimney = higher efficiency.
Flue gas is always moisture-laden, and starting at a low temperature and then dumping into a cold masonry chimney means the moisture will condense out inside the chimney, rather than exiting as a gas and condensing outside the chimney. Depending on the gas quality, this condensate may be somewhat acidic, thus corrosive (attacks mortar, steel etc.)
It would be worth checking your furnace manual for chimney recommendations - many of the very high efficiency units actually want to be vented into PVC that's sloped to let the moisture drain out (or at least has a drain at the bottom, if vertical.) The temperature (on those furnaces) is so low that PVC is perfectly adequate for the temperature, and it has good chemical resistance. A stainless steel liner would be another possible option - a masonry liner might well not. In any case, what you use should comply with what the new furnace wants, which may not be what the old furnace had (or needed.)


Is all RAW data preserved when editing photo in Photoshop?

Question: It is possible to edit RAW ("undeveloped") photos in Adobe Camera Raw but what if I open the photo in Photoshop's main windows and then apply the Camera Raw Filter instead? Do I still have all the data from the original undeveloped shot to work with? Or does something get lost and it's better to edit in ACR first before opening in the main editing window?
Answer: Everything you do to a raw file is recorded as a set of instructions on how to interpret the original raw image data, which does not change (assuming the file doesn't suffer random data corruption in whatever storage medium it is held, or whenever it is copied to another storage medium) as long as the original file is retained.
Those instruction sets are generally proprietary to whatever application was used to create them. Work you do in Adobe products will not be applied if you later open the raw image file in other raw conversion applications, such as Capture One or DxO PhotoLab (formerly Dxo Optics Pro). Similarly, any work you do in Capture One will not be reflected when the raw image file is later opened using Adobe Camera Raw.
In the case of Adobe products, Adobe Camera Raw is used to interpret raw image files when worked with in other Adobe applications such as Adobe Photoshop or Adobe Lightroom. So whether you open the file in Adobe Camera Raw or in Adobe Photoshop, the raw conversion engine under the hood is the same: ACR. The interface is all that is different with regard to how raw files are handled in Photoshop. The additional functions/options of Photoshop that are not included in ACR are applied to the image information after it has been converted from raw image data to a raster image by ACR working in the background, and those instructions are also added to the file (or a sidecar file containing the instructions) without altering any of the original raw image data.
Changing the way an image looks on your screen using ACR, LR, or PS doesn't alter the original file at all until you "save" it. It just changes what you see on the screen by changing the way the raw image information is processed/interpreted. When you "save" your work, the instructions are added to either the image file itself or to a separate sidecar file associated with the image file. (There are options within ACR and all the other Adobe products that use ACR to interpret raw data that allows the end user to select whether the instruction set is appended to the main file or contained in a sidecar file.) In either case, none of the original image data is replaced. The instructions on how the original data is to be interpreted are added and used again the next time the file is opened using an Adobe product. If you "save" your changes, make additional changes and then "save" it again, the latest instruction set will overwrite the previous instructions
with regard to the particular changes you made.
One analogy is that of a culinary dish. The raw image data is like the list of ingredients. The instruction set is like the various instructions on how to prepare those ingredients. One can take the same ingredients and prepare them differently and get two different results. One can even use only some (but not all) of the ingredients on the list to get various results. One might have one set of instructions for preparing the dish in a conventional oven, another set of instructions for preparing the dish using a microwave oven, and yet another set of instructions for using an open fire and a dutch oven. Changing the instructions on how to prepare them does not change the initial set of ingredients we start with. It does change, to one degree or another, what we end up eating.
However, a raw image file usually contains more information than just the image data collected from the sensor. Much of this information is stored in a portion of the file known as the EXIF information. Much of the EXIF info is standardized, but there are provisions for what are known as "maker notes" to also be included in the EXIF information. These "maker notes" allow each camera manufacturer to include whatever non-standardized information they desire in the EXIF info. For the most part, Adobe products ignore most of the maker notes section of the EXIF info, but other raw conversion applications, particularly those created by the camera manufacturer like Nikon's ViewNX or Canon's Digital Photo Professional, can and often do use some of that information when interpreting the raw image data.
When images are exported using Adobe products, much of the maker notes section included in the original raw file is not included in the EXIF info of the new file created by Adobe products. It's still there in the original raw image file unless the original is overwritten by the new exported file. Again, there are options from within Adobe's products to leave the original file and create another exported file or to replace the original file with the exported file.
For example, if one converts a Nikon .NEF file to .DNG using Adobe's convertor, the maker notes are stripped from the EXIF information and not included in the resulting DNG. If one only plans to ever use Adobe products this is no big deal, because all Adobe products ignore the information that is stripped. But if one later wants to use another raw conversion application that does use the maker notes info, that information is gone unless one still has the original .NEF as well as the .DNG made from it.
Comment: That's good to know. However, the vast majority of folks who convert various raw formats to DNG usually do not select the option to include all of the original file because of file size considerations. One of the benefits touted by Adobe when they introduced the DNG format was that it reduced file sizes and that is why many folks decided to use it instead of keeping their actual raw files as they came out of their cameras.
Comment: One small clarification: DNG convertor have option to include original RAW file in resulting DNG so you can restore the original RAW even if you do not keep it as separate file
Comment: It is my understanding that it still strips most of the "maker notes" section of the EXIF info, though it does preserve the actual data from the sensor.
Comment: Just checked and (for Canon CR2) DNG Convertor restore 100% original file (checked with sha1 hash)
Answer: I all the cases you have untouched RAW file. All the changes (lossless) are stored in special area of the file (when file is DNG), stored in side file (XMP) or in special catalog (in Lightroom for example).
And you can get back untouched RAW file just by delete this side file. Or by reset settings you done.
Answer: The following edits are non-destructive when applied to a raw file:

any edit in Lightroom
any edit in Camera Raw
any edit in Photshop using adjustment layers

Almost all other edits in Photoshop (filters, image adjustments, etc.) are destructive. If you overwrite the original file with these edits, your original data are lost. Fortunately, you can't save to a raw file format (it wouldn't be raw anymore), so you can't overwrite a raw file (but you can overwrite a jpeg or any other file format and your original data will be lost).
A good practice when working in Photoshop is to open a raw, edit it in Camera Raw and then edit your image and save it as a psd (PhotoShop Document).
In Lightroom and Camera Raw, your edits are not stored in the raw file, but in the (Lightroom) library or in separate (xmp) files, which just contain the edits to the original (raw) image. When opening such a file, Lightroom/Photoshop reads your image and re-applies those edits.
Comment: In PS you can't save the edited file to RAW. You can save it to image file. Independently if you do destructive or nondestructive edits
Comment: @RomeoNinov thanks. Wasn't sure if PS could do that or not. Corrected the answer accordingly.
Comment: Has Adobe removed the option to append the development instruction set to the original raw file in the most recent versions of ACR/PS/LR? It was there at one time, though not too many folks used it because it meant needing to rewrite the entire file each time it was saved and that increased the risk of data corruption as well as took longer. With newer OSs, though, files can be amended with additional information without having to rewrite the entire file.
Comment: @MichaelC, last LR have option to add date/time of change to RAW. And can add development settings only in JPEG, TIFF, PSD and PNG. Not sure about PS/ACR
Answer: The camera raw filter inside PS is not using the original raw data, nor the raw edits file (xmp).
You can verify this by editing the image in ACR before opening it in Photoshop and then applying the camera raw filter.
If the filter was using the raw data/xmp ACR would open the image with the original ACR edits shown (slider positions); in the same manner that LR displays a copy, and how ACR opens a raw file with an associated xmp. But instead the filter opens the image with the previous ACR edits applied, as well as any (destructive) edits that were made to the layer inside PS (all sliders/edits at zero).
The xmp file is created by ACR when you originally open/edit the file. Edits made w/in photoshop are made using that modified data. And that includes edits made with the camera raw filter... i.e. those PS filter edits are not added to the xmp because they are not raw file edits (instructions).
In other words, ACR is destructive in terms of the image/data used w/in PS. If your original ACR edits are bad you will not be able to restore the image unless you start over from the original raw file... in which case it will open in ACR with your previous "bad edits" showing, and you can adjust them to a new positions as desired.
I.e. if you apply a +5 exposure in ACR when you first open the raw file, the camera raw filter will subsequently open the image with the exposure set to zero, and a -5 adjustment will not restore it to the original raw state. To get back to the original raw state with the exposure slider at +5 (resettable) you will need to start from the raw file again.


OS Deployment via PXE - Preparing and Using Images in Many Small Environments (MSP)

Question: I've got a project going on which I'm running into a bit of a roadblock on.
I'm interested in learning if there is a better alternative to what my MSP does for OS deployment, currently. Most of our clients are small (10-20) seat environments - we try to keep each workstation identical for the most part. Currently, when a computer needs to be wiped and restored, we walk the client through taking an image of a healthy computer on Acronis, and putting it onto the first machine. Then we change the host names, switch the key codes, and disjoin / rejoin it to the domain (the computer being off during this whole exchange, up to the rejoin). These would be Windows XP and 7 machines, with versions of Office from 2003 to 2013 plus proprietary industry software.
What I'd like to do is see if there's a better way to do this. In particular, I'm interested in keeping the need for end-user interaction to a minimum. We set client computers to have PXE boot enabled, so PXE boot solutions are possible. The first major problem I have to overcome is efficiency - our clients have disparate enough situations that it isn't feasible to just have a few images for the entire base of them. A favorable aspect of the Acronis solution is that we can get it pretty close to the set of applications each client needs. Initially I was thinking to sysprep a machine at each client's location, but after further research I realized this would be a very bad idea due to wiping the sysprep'd computer.
I would especially like it to be possible to seize an image of a computer on the network without needing to kick the end user off of it, possibly via PsExec or PowerShell, prep it, and deliver to the new machine, with the absolute minimum amount of end user effort. I have a lab I can investigate solutions in.
Currently, the most promising solution is appearing to be FOG, but from what I can tell in the documentation, I would still need to prep an image from a computer on the network via sysprep, thus removing some of the efficiency from this equation (and needing to kick the user off). Is it possible to use PsExec to remotely run Disk2VHD, and then sysprep /generalize the VHD for deployment to physical machines?
Ultimately, what makes this question odd is that almost every image-based tutorial and resource assumes you're doing a lot of machines which will be similar, which is a disparate use case for what I would like this to work for. I know this is a very complicated question - thanks in advance if you help!
Answer: When I've dealt with this kind of situation I would create a clone of a base system - I preferred using vmwares p2v and workstation for the snapshotting capability. I'd then take the clone and use it to sysprep and generalize an image I could deploy with MDT. I'd include the drivers for the physical hardware being deployed to in the task sequence.
It isn't very scriptable doing it this way but it does avoidyhr need if taking systems offline.