Strengthen auto generation warnings.

Strengthen auto generation warnings.

# combine various bits to create a single documentation file

redo-ifchange $2.txt

if test $2 = "redo" ; then
  name=$2
else
  name="redo-"$2
fi

cat <<EOS
% $name(1) Redux User Manual
% Gyepi Sam
% $(date +'%B %d, %Y')

<!-- DO NOT EDIT -- Autogenerated file. Really! -->

EOS

../bin/redux documentation $2 | sed 's/^\( \+-.\+\)/\1\n/'

echo

cat $2.txt

# combine various bits to create a single documentation file

redo-ifchange $2.txt

if test $2 = "redo" ; then
  name=$2
else
  name="redo-"$2
fi

cat <<EOS
% $name(1) Redux User Manual
% Gyepi Sam
% $(date +'%B %d, %Y')

<!-- DO NOT EDIT -- Autogenerated file -->

EOS

../bin/redux documentation $2 | sed 's/^\( \+-.\+\)/\1\n/'

echo

cat $2.txt


Add explicit shebang

Add explicit shebang

#!/bin/sh -e
# define all project wide variables

export ROOTDIR=$(pwd)
export SRCDIR=$ROOTDIR/src
export OUTDIR=$ROOTDIR/out
export VERSION="pre-0.01"

DESTDIR=${DESTDIR-/usr/local/bin}

if [ "$1" = "all" ]; then
	redo-ifchange "$OUTDIR/redo"
elif [ "$1" = "clean" ]; then
	rm -rf "$OUTDIR"/*.tmp "$OUTDIR"/*.o "$OUTDIR"/redo "$OUTDIR"/CC
	# autoconf stuff
	rm -rf autom4te.cache config.h.in configure config.status config.log config.h
elif [ "$1" = "install" ]; then
	redo-ifchange all
	mkdir -p "$DESTDIR"
	install "$OUTDIR/redo" "$DESTDIR"
	ln -sf "$DESTDIR/redo" "$DESTDIR/redo-ifchange"
	ln -sf "$DESTDIR/redo" "$DESTDIR/redo-ifcreate"
	ln -sf "$DESTDIR/redo" "$DESTDIR/redo-always"
	echo "Finished installing."
fi


ui-macos/bits/runpython.do: skip ppc64 architecture.

I don't have a Mac that can build it.  Hopefully ppc will run fine on ppc64.

exec >&2
redo-ifchange runpython.c
ARCHES=""
printf "Platforms: "
for d in /usr/libexec/gcc/darwin/*; do
    PLAT=$(basename "$d")
    [ "$PLAT" != "ppc64" ] || continue  # fails for some reason on my Mac
    ARCHES="$ARCHES -arch $PLAT"
    printf "$PLAT "
done
printf "\n"
gcc $ARCHES \
	-Wall -o $3 runpython.c \
	-I/usr/include/python2.5 \
	-lpython2.5

exec >&2
redo-ifchange runpython.c
ARCHES=""
printf "Platforms: "
for d in /usr/libexec/gcc/darwin/*; do
    PLAT=$(basename "$d")
    ARCHES="$ARCHES -arch $PLAT"
    printf "$PLAT "
done
printf "\n"
gcc $ARCHES \
	-Wall -o $3 runpython.c \
	-I/usr/include/python2.5 \
	-lpython2.5


test.do: allow docs to build in parallel with tests.

Previously, we'd try to build all the critical stuff first, and then
run the tests.  Nowadays, it takes a little longer to build the docs
(especially some of the docs/cookbook/ stuff), and this isn't needed to
run the tests, so let's allow them to parallelize.

redo-ifchange bin/all
redo minimal/test docs/test t/all _all
[ -n "$DO_BUILT" ] || echo "Don't forget to test 'minimal/do -c test'" >&2

redo-ifchange _all
redo minimal/test docs/test t/all
[ -n "$DO_BUILT" ] || echo "Don't forget to test 'minimal/do -c test'" >&2


Push from cygnus Sun Apr  9 22:13:54 PDT 2017

= Honu: to.do: =

  o bloggish directories should have a "make wc" target that subtracts the word count of
    the template file.  Or, better, use sed to filter out tags.

  o bootstrap should check ~/.ssh to see whether to use savitzky.net or github.

  o Much of the configuration should be refactored to put more of the customization into
    Myrtle/Mathilda; Honu should become more of a framework.  A similar structure might
    work for MakeStuff.

  o Some of the configuration is obsolete, and is unlikely ever to be used again.

========================================================================================>
Work Log:
========

2015:
=====

1111We
  @ OpenSSH/Cookbook/Multiplexing - Wikibooks
    Makes ssh much faster, and circumvents dreamhost's connection limits.
    > mkdir ~/.ssh/controlmasters; chmod 700 ~/.ssh/controlmasters
  * Add the following to .ssh/config, in the global section at the top:
      ControlPath ~/.ssh/controlmasters/%r@%h:%p
      ControlPersist 10m
      ControlMaster auto
  @ https://github.com/blipvert/ljupdate/commits/master marginally more up to date
    * update emacs/depends.make

2016:
=====

0214Su
  * much hacking in gnome-setup and font setup over the last week or so.

0215Mo
  * added adobe-source-code-pro font, tell emacs to use system font.

... Really ought to copy the config-related stuff out of starport/Admin/admin.to.do

0626
  ~ alltray looked useful - lets you put an icon for any program into the tray.
    It starts the program if it's not already running, but fails in xmonad because it's
    waiting for the program to be iconified.
  * gsimplecal, fbpanel, and probably others have subdirectories in .config
    probably best is to make another directory called dotconfig, parallel to dotfiles

0627Mo
  * made the Makefiles independendent of Tools/Makefile.

0704Mo
  * git subtree add --prefix bin ../bin master
  * The idea is that if there's a pre-existing ~/bin, we can symlink into it.

0705Tu
  * regular dotfiles should start with "_" rather than ".", to make them visible.
    -> move dotfiles/.* -> dotfiles/_*; update dotfiles/Makefile

0716Sa
  . Some dotfiles may need to be modified by a script rather than simply symlinked.
    That's because things like username and email address change from one system to
    another.  In most cases, though, one can use include files generated by make install.

0719Tu
  * When site-dependent parameters can be in separate files (include files/modules).
    the various install targets can initialize them from templates.  A good place to start
    would be .xmonad/Local.hs.  Could also do this for .bashrc; a conditional include
    would make it unnecessary to generate the file if it's not needed. (0716)
    : for gitconfig - note that the file doesn't need to exist
      [include]
	  path = /path/to/file

0725Mo
  * color-coded workspace labels.  ws N makes an xclock with WS=N and the appropriate
    resistor color code.
  * the ability to make ad-hoc workspace labels might help.  Put the list in a file that
    can be read in either xmonad or the bottom-bars script.  Possibly a top dbar with
    vertical menu. (0719) -> add a second argument to ws.
  * Re-running ws N should grep ps for the appropriate color code and kill it.
    An alternative to xclock would be dzen -p.

0726Tu
  x use dzen for workspace labels?  Note that dzen is smaller, but only slightly.
    Problem is that it doesn't seem possible to have it shrinkwrap text.
  * separate widths for first and other top bars; make default 950; adjust ws

0827Sa
  * Allow MakeStuff as well as Tools.  Eventually it will be possible to drop Tools as
    an alternative.

0828Su
  ^ Config -> Honu.  The only real question is whether I want to make Honu a separate
    version, cleaned up and generalized for distribution. -> I don't think so.  It would
    be hard to keep them in sync.  The main me-specific things are the link pages in web,
    and some of the stuff in Archive, but there are also some of the dotfiles (e.g.
    .signature and .Xdefaults-*), stuff in the emacs config, and...  On second thought: I
    think I'd be a very uncomfortable about putting .mailrc, .gnus, and the bookmarks out
    there.  So there's going to have to be a rewrite anyway.  Those can go in -Personal-
  ^ Problematic stuff -- bookmarks, .mailrc, .gnus, etc. belongs in a separate git repo
    that's a subdirectory of local!  Can of course have dotfiles and dotconfig subdirs.
    Can have its own bootstrap, which would get the user-specific out of there, too.  In
    fact, call it local/local-Config or local/My-Honu or some such and derive it from the
    original, with all the old history.  Make a relatively clean start for Honu.

0829Mo
  o bootstrap and the first run of make should log messages, and print out the log on
    exit.

0904Su
  * clone from Config; clean up with bfg:
    78  alias bfg='java -jar ~/Downloads/bfg-1.12.13.jar'
    85  bfg --help
    87  git commit -m "deletions before bfg"
    88  bfg --delete-folders Archive
    91  bfg --delete-folders Archive
    92  bfg --delete-folders web
    94  bfg -D [._]mailrc
    95  bfg -D [._]gnus
    96  bfg -D [._]sig*
    97  bfg -D [hw].home.html
   107  git filter-branch --prune-empty
   110  git reflog expire --expire=now --all && git gc --prune=now --aggressive

0905Mo
  * Push to github.
  o .emacs:  generate, and load Honu/emacs/emacs.el and Honu/local/emacs.el
  o testing is needed:
    Set up a testing environment by setting USER and HOME, and putting a dummy sudo
    that simply records what it's supposed to do on $PATH.  Ought to have a dummy groups
    command as well. -> see MakeStuff for test framework.
  o Originals:  Should go into, e.g., local/orig  (ignore subdirectories of local)
  * Myrtle/Mathilda - install into the same directory as Honu and MakeStuff.
    symlink files and subdirs into Honu/local, or directly into $HOME if they don't
    overlap.
  o add scsitools to packages

0911Su
  * no links to bottom-bars, bottom-mobars
  o length on bottom-bars -- see ws for how to get this

0912Mo
  ^ At some point, we may want to move some of the settings most likely to be customized
    out into Myrtle/Mathilda.  That includes whether to pull down MakeStuff -- Honu should
    be completely stand-alone and not need much customization.

1226Mo
  ~ use roxterm as the preferred terminal rather than gnome-terminal.
    Can use roxterm --separate --name=wsN with wssetup

2017:
=====

0128Sa
  : roxterm doesn't work as well as I'd like; the profile is split up and some parts can't
    have their defaults configured (in particular whether to show the tab bar when there's
    only one tab present, which is the usual case the way I do it.

0408Sa
  o add alt-sends-escape to XTerm config

========================================================================================>|
Local Variables:
    fill-column:90
End:

= Honu: to.do: =

  o bloggish directories should have a "make wc" target that subtracts the word count of
    the template file.  Or, better, use sed to filter out tags.

  o bootstrap should check ~/.ssh to see whether to use savitzky.net or github.

  o Much of the configuration should be refactored to put more of the customization into
    Myrtle/Mathilda; Honu should become more of a framework.  A similar structure might
    work for MakeStuff.

  o Some of the configuration is obsolete, and is unlikely ever to be used again.

========================================================================================>
Work Log:
========

2015:
=====

1111We
  @ OpenSSH/Cookbook/Multiplexing - Wikibooks
    Makes ssh much faster, and circumvents dreamhost's connection limits.
    > mkdir ~/.ssh/controlmasters; chmod 700 ~/.ssh/controlmasters
  * Add the following to .ssh/config, in the global section at the top:
      ControlPath ~/.ssh/controlmasters/%r@%h:%p
      ControlPersist 10m
      ControlMaster auto
  @ https://github.com/blipvert/ljupdate/commits/master marginally more up to date
    * update emacs/depends.make

2016:
=====

0214Su
  * much hacking in gnome-setup and font setup over the last week or so.

0215Mo
  * added adobe-source-code-pro font, tell emacs to use system font.

... Really ought to copy the config-related stuff out of starport/Admin/admin.to.do

0626
  ~ alltray looked useful - lets you put an icon for any program into the tray.
    It starts the program if it's not already running, but fails in xmonad because it's
    waiting for the program to be iconified.
  * gsimplecal, fbpanel, and probably others have subdirectories in .config
    probably best is to make another directory called dotconfig, parallel to dotfiles

0627Mo
  * made the Makefiles independendent of Tools/Makefile.

0704Mo
  * git subtree add --prefix bin ../bin master
  * The idea is that if there's a pre-existing ~/bin, we can symlink into it.

0705Tu
  * regular dotfiles should start with "_" rather than ".", to make them visible.
    -> move dotfiles/.* -> dotfiles/_*; update dotfiles/Makefile

0716Sa
  . Some dotfiles may need to be modified by a script rather than simply symlinked.
    That's because things like username and email address change from one system to
    another.  In most cases, though, one can use include files generated by make install.

0719Tu
  * When site-dependent parameters can be in separate files (include files/modules).
    the various install targets can initialize them from templates.  A good place to start
    would be .xmonad/Local.hs.  Could also do this for .bashrc; a conditional include
    would make it unnecessary to generate the file if it's not needed. (0716)
    : for gitconfig - note that the file doesn't need to exist
      [include]
	  path = /path/to/file

0725Mo
  * color-coded workspace labels.  ws N makes an xclock with WS=N and the appropriate
    resistor color code.
  * the ability to make ad-hoc workspace labels might help.  Put the list in a file that
    can be read in either xmonad or the bottom-bars script.  Possibly a top dbar with
    vertical menu. (0719) -> add a second argument to ws.
  * Re-running ws N should grep ps for the appropriate color code and kill it.
    An alternative to xclock would be dzen -p.

0726Tu
  x use dzen for workspace labels?  Note that dzen is smaller, but only slightly.
    Problem is that it doesn't seem possible to have it shrinkwrap text.
  * separate widths for first and other top bars; make default 950; adjust ws

0827Sa
  * Allow MakeStuff as well as Tools.  Eventually it will be possible to drop Tools as
    an alternative.

0828Su
  ^ Config -> Honu.  The only real question is whether I want to make Honu a separate
    version, cleaned up and generalized for distribution. -> I don't think so.  It would
    be hard to keep them in sync.  The main me-specific things are the link pages in web,
    and some of the stuff in Archive, but there are also some of the dotfiles (e.g.
    .signature and .Xdefaults-*), stuff in the emacs config, and...  On second thought: I
    think I'd be a very uncomfortable about putting .mailrc, .gnus, and the bookmarks out
    there.  So there's going to have to be a rewrite anyway.  Those can go in -Personal-
  ^ Problematic stuff -- bookmarks, .mailrc, .gnus, etc. belongs in a separate git repo
    that's a subdirectory of local!  Can of course have dotfiles and dotconfig subdirs.
    Can have its own bootstrap, which would get the user-specific out of there, too.  In
    fact, call it local/local-Config or local/My-Honu or some such and derive it from the
    original, with all the old history.  Make a relatively clean start for Honu.

0829Mo
  o bootstrap and the first run of make should log messages, and print out the log on
    exit.

0904Su
  * clone from Config; clean up with bfg:
    78  alias bfg='java -jar ~/Downloads/bfg-1.12.13.jar'
    85  bfg --help
    87  git commit -m "deletions before bfg"
    88  bfg --delete-folders Archive
    91  bfg --delete-folders Archive
    92  bfg --delete-folders web
    94  bfg -D [._]mailrc
    95  bfg -D [._]gnus
    96  bfg -D [._]sig*
    97  bfg -D [hw].home.html
   107  git filter-branch --prune-empty
   110  git reflog expire --expire=now --all && git gc --prune=now --aggressive

0905Mo
  * Push to github.
  o .emacs:  generate, and load Honu/emacs/emacs.el and Honu/local/emacs.el
  o testing is needed:
    Set up a testing environment by setting USER and HOME, and putting a dummy sudo
    that simply records what it's supposed to do on $PATH.  Ought to have a dummy groups
    command as well. -> see MakeStuff for test framework.
  o Originals:  Should go into, e.g., local/orig  (ignore subdirectories of local)
  * Myrtle/Mathilda - install into the same directory as Honu and MakeStuff.
    symlink files and subdirs into Honu/local, or directly into $HOME if they don't
    overlap.
  o add scsitools to packages

0911Su
  * no links to bottom-bars, bottom-mobars
  o length on bottom-bars -- see ws for how to get this

0912Mo
  ^ At some point, we may want to move some of the settings most likely to be customized
    out into Myrtle/Mathilda.  That includes whether to pull down MakeStuff -- Honu should
    be completely stand-alone and not need much customization.

1226Mo
  ~ use roxterm as the preferred terminal rather than gnome-terminal.
    Can use roxterm --separate --name=wsN with wssetup

2017:
=====

0128Sa
  : roxterm doesn't work as well as I'd like; the profile is split up and some parts can't
    have their defaults configured (in particular whether to show the tab bar when there's
    only one tab present, which is the usual case the way I do it.

========================================================================================>|
Local Variables:
    fill-column:90
End:


Fix regression caused by 5e5e7e6

Fix regression caused by 5e5e7e6

We need to use the source operator on the CC script as if we don't
bad return codes won't get forwarded and the script won't get aborted.


Line fixes.

Line fixes.

* Call OpenIRT
*
capture program drop openirt
program  openirt
  version 10.1
	preserve
	set more off
	* set eolchar unix
	local seed = round(uniform()*10000000)
	syntax, id(varname) item_prefix(name) save_item_parameters(string) save_trait_parameters(string) ///
		[samplesize(integer 2000) burnin(integer 1000) theta(varname) model(string) ///
		fixed_item_file(string)]

	isid `id'
	aorder `item_prefix'*

	* List selected items
	display as text "Selected items: " _continue
	foreach v of varlist `item_prefix'* {
		local itemid : subinstr local v "`item_prefix'" "", all
		local item_list `item_list' `itemid'
	}
	display "`item_list'"

	* Write out response data
	display as text "Setting up response data..."
	tempfile response_file
	tempvar group

	if("`theta'" == "") {
		tempvar theta
		qui gen `theta' = -9999
	}
	else {
		qui count if theta != .
		display as text "Number of fixed theta: `r(N)'"
	}

	* check item responses
	foreach v of varlist `item_prefix'* {
		qui sum `v'
		if(r(min)<0 | r(max) > 1) {
			display as error "`v': Response data must be coded 0/1."
			exit
		}
		if(r(mean)==0) {
			display as error "Warning: `v' is 100 percent 0, consider dropping item."
		}
		if(r(mean)==1) {
			display as error "Warning: `v' is 100 percent 1, consider dropping item."
		}
	}
	qui recode _all (.=-9999)
	qui gen `group' = 1
	qui outsheet `id' `group' `theta' `item_prefix'* using openirt_tmp.txt, replace delim(" ") noquote nolabel nonames
	capture erase `response_file'
	filefilter openirt_tmp.txt `response_file', from("\r\n") to("\n")
	capture erase openirt_tmp.txt

	* Write out parameter data
	display as text "Setting up parameter data..."
	clear
	local num_items : list sizeof item_list
	qui set obs `num_items'
	qui gen id = .
	local i = 1
	foreach	id of local item_list {
		qui replace id = `id' in `i'
		local ++i
	}
	if("`model'" == "2PL") {
		gen type = 1
		display as text "Default model: Two parameter logistic (2PL)"
	}
	else {
		gen type = 2
		display as text "Default model: Three parameter logistic (3PL)"
	}
	gen numcat = 2
	gen a = -9999
	gen b = -9999
	gen c = -9999
	gen d1 = -9999
	gen d2 = -9999
	gen d3 = -9999
	gen d4 = -9999
	tempfile item_file
	sort id
	qui save `item_file', replace

	if("`fixed_item_file'" != "") {
		cap use `fixed_item_file', clear
		if(_rc != 0) {
			display as error "Could not open fixed item parameter file: `fixed_item_file'."
			exit
		}
		confirm variable id
		confirm variable type
		confirm variable a
		confirm variable b
		confirm variable c
		keep id type a b c
		sort id
		qui merge id using `item_file', update
		qui count if _m==1
		if(`r(N)' > 0) {
			display as error "Fixed item ids do not match response item ids."
			exit
		}
	}
	qui compress
	sort id
	isid id
	qui outsheet id type numcat a b c d1 d2 d3 d4 using openirt_tmp.txt, replace delim(" ") noquote nolabel nonames
	capture erase `item_file'
	filefilter openirt_tmp.txt `item_file', from("\r\n") to("\n")
	capture erase openirt_tmp.txt

	* Run estimation routine from shell
	qui findfile openirt.exe
	local execfile `r(fn)'
	local execfile : subinstr local execfile " " "\ ", all
	!chmod +x `execfile'

	qui findfile openirt.ini

	*local inifile : subinstr local inifile " " "\ ", all
	*display "`inifile'"
	tempfile inifile
	qui copy "`r(fn)'" `inifile', replace
	tempfile testout
	tempfile responseout
	capture erase `reponseout'
	capture erase `testout'
	local a `execfile' --config-file=`inifile' --test-file=`item_file' --response-file=`response_file' ///
    --test-outfile=`testout' --response-outfile=`responseout' ///
    --sample-size=`samplesize' --burnin=`burnin' --thin=1 --random-seed=`seed'
	* display "`a'"
	! `a'

	cap confirm file `responseout'
	if(_rc == 601) {
		display as error "OpenIRT could not estimate your model.  Please check the specified options and try again."
		exit
	}

	* Test parameters
	qui insheet using `testout', clear
	lab var id "Item ID"
	lab var type "Item Type"
	lab define itemtype 1"2PL" 2"3PL"
	lab val type itemtype
	lab var a_eap "Item discrimination (EAP)"
	lab var b_eap "Item difficulty (EAP)"
	lab var c_eap "Item difficulty (EAP)"
	forvalues i = 1/5 {
		lab var a_pv`i' "Item discrimination (Plausible value `i')"
		lab var b_pv`i' "Item difficulty (Plausible value `i')"
		lab var c_pv`i' "Item guessing (Plausible value `i')"
	}
	qui compress
	qui recode _all (-9999=.)
	sort id
	display as text _newline "Saving item parameters..."
	save `save_item_parameters', replace

	qui insheet using `responseout', clear
	lab var id "Respondent ID"
	drop group
	lab var theta_eap "Theta (EAP)"
	forvalues i = 1/5 {
		lab var theta_pv`i' "Theta (Plausible value `i')"
	}
	lab var theta_mle "Theta (MLE)"
	lab var theta_mle_se "Theta standard error (MLE)"

	* some standard errors may be nan if likelihood very flat.  replace with missing
	qui destring _all, force replace
	qui recode _all (-9999=.)

	display as text _newline "Saving traits..."
	qui compress
	sort id
	save `save_trait_parameters', replace
	restore
	set more on
end

* net install /Users/tristanz/Data/OpenIRT/Stata/openirt, force
* sysuse naep_children, clear
* openirt, id(id) item_prefix(item) save_item_parameters(items) save_trait_parameters(traits)

* Call OpenIRT
*
capture program drop openirt
program  openirt
  version 10.1
	preserve
	set more off
	* set eolchar unix
	local seed = round(uniform()*10000000)
	syntax, id(varname) item_prefix(name) save_item_parameters(string) save_trait_parameters(string) ///
		[samplesize(integer 2000) burnin(integer 1000) theta(varname) model(string) ///
		fixed_item_file(string)]

	isid `id'
	aorder `item_prefix'*

	* List selected items
	display as text "Selected items: " _continue
	foreach v of varlist `item_prefix'* {
		local itemid : subinstr local v "`item_prefix'" "", all
		local item_list `item_list' `itemid'
	}
	display "`item_list'"

	* Write out response data
	display as text "Setting up response data..."
	tempfile response_file
	tempvar group

	if("`theta'" == "") {
		tempvar theta
		qui gen `theta' = -9999
	}
	else {
		qui count if theta != .
		display as text "Number of fixed theta: `r(N)'"
	}

	* check item responses
	foreach v of varlist `item_prefix'* {
		qui sum `v'
		if(r(min)<0 | r(max) > 1) {
			display as error "`v': Response data must be coded 0/1."
			exit
		}
		if(r(mean)==0) {
			display as error "Warning: `v' is 100 percent 0, consider dropping item."
		}
		if(r(mean)==1) {
			display as error "Warning: `v' is 100 percent 1, consider dropping item."
		}
	}
	qui recode _all (.=-9999)
	qui gen `group' = 1
	qui outsheet `id' `group' `theta' `item_prefix'* using openirt_tmp.txt, replace delim(" ") noquote nolabel nonames
	filefilter openirt_tmp.txt `response_file', from("\r\n") to("\n")
	erase openirt_tmp.txt

	* Write out parameter data
	display as text "Setting up parameter data..."
	clear
	local num_items : list sizeof item_list
	qui set obs `num_items'
	qui gen id = .
	local i = 1
	foreach	id of local item_list {
		qui replace id = `id' in `i'
		local ++i
	}
	if("`model'" == "2PL") {
		gen type = 1
		display as text "Default model: Two parameter logistic (2PL)"
	}
	else {
		gen type = 2
		display as text "Default model: Three parameter logistic (3PL)"
	}
	gen numcat = 2
	gen a = -9999
	gen b = -9999
	gen c = -9999
	gen d1 = -9999
	gen d2 = -9999
	gen d3 = -9999
	gen d4 = -9999
	tempfile item_file
	sort id
	qui save `item_file', replace

	if("`fixed_item_file'" != "") {
		cap use `fixed_item_file', clear
		if(_rc != 0) {
			display as error "Could not open fixed item parameter file: `fixed_item_file'."
			exit
		}
		confirm variable id
		confirm variable type
		confirm variable a
		confirm variable b
		confirm variable c
		keep id type a b c
		sort id
		qui merge id using `item_file', update
		qui count if _m==1
		if(`r(N)' > 0) {
			display as error "Fixed item ids do not match response item ids."
			exit
		}
	}
	qui compress
	sort id
	isid id
	qui outsheet id type numcat a b c d1 d2 d3 d4 using openirt_tmp.txt, replace delim(" ") noquote nolabel nonames
	filefilter openirt_tmp.txt `item_file', from("\r\n") to("\n")
	erase openirt_tmp.txt

	* Run estimation routine from shell
	qui findfile openirt.exe
	local execfile `r(fn)'
	local execfile : subinstr local execfile " " "\ ", all
	!chmod +x `execfile'

	qui findfile openirt.ini

	*local inifile : subinstr local inifile " " "\ ", all
	*display "`inifile'"
	tempfile inifile
	qui copy "`r(fn)'" `inifile', replace
	tempfile testout
	tempfile responseout
	capture erase `reponseout'
	capture erase `testout'
	local a `execfile' --config-file=`inifile' --test-file=`item_file' --response-file=`response_file' ///
    --test-outfile=`testout' --response-outfile=`responseout' ///
    --sample-size=`samplesize' --burnin=`burnin' --thin=1 --random-seed=`seed'
	* display "`a'"
	! `a'

	cap confirm file `responseout'
	if(_rc == 601) {
		display as error "OpenIRT could not estimate your model.  Please check the specified options and try again."
		exit
	}

	* Test parameters
	qui insheet using `testout', clear
	lab var id "Item ID"
	lab var type "Item Type"
	lab define itemtype 1"2PL" 2"3PL"
	lab val type itemtype
	lab var a_eap "Item discrimination (EAP)"
	lab var b_eap "Item difficulty (EAP)"
	lab var c_eap "Item difficulty (EAP)"
	forvalues i = 1/5 {
		lab var a_pv`i' "Item discrimination (Plausible value `i')"
		lab var b_pv`i' "Item difficulty (Plausible value `i')"
		lab var c_pv`i' "Item guessing (Plausible value `i')"
	}
	qui compress
	qui recode _all (-9999=.)
	sort id
	display as text _newline "Saving item parameters..."
	save `save_item_parameters', replace

	qui insheet using `responseout', clear
	lab var id "Respondent ID"
	drop group
	lab var theta_eap "Theta (EAP)"
	forvalues i = 1/5 {
		lab var theta_pv`i' "Theta (Plausible value `i')"
	}
	lab var theta_mle "Theta (MLE)"
	lab var theta_mle_se "Theta standard error (MLE)"

	* some standard errors may be nan if likelihood very flat.  replace with missing
	qui destring _all, force replace
	qui recode _all (-9999=.)

	display as text _newline "Saving traits..."
	qui compress
	sort id
	save `save_trait_parameters', replace
	restore
	set more on
end

* net install /Users/tristanz/Data/OpenIRT/Stata/openirt, force
* sysuse naep_children, clear
* openirt, id(id) item_prefix(item) save_item_parameters(items) save_trait_parameters(traits)


iegitaddmd : helpfile update issue #118

iegitaddmd : helpfile update issue #118

{smcl}
{* 15 Dec 2017}{...}
{hline}
help for {hi:iegitaddmd}
{hline}

{title:Title}

{phang}{cmdab:iegitaddmd} {hline 2} Creates a placeholder file in subfolders of a GitHub repository folder, which allows to commit folder structures with empty folders.

{title:Syntax}

{phang} {cmdab:iegitaddmd} , {cmd:folder(}{it:file_path}{cmd:)} [file({help filename}) all skip replace]

{marker opts}{...}
{synoptset 18}{...}
{synopthdr:options}
{synoptline}
{synopt :{cmd:folder(}{it:file_path}{cmd:)}}Specifies the folder path to the repository folder{p_end}
{synopt :{cmd:file(}{it:{help filename}}{cmd:)}}Specifies a file saved on disk that is used instead of the default file as placeholder{p_end}
{synopt :{cmd:all}}Creates the placeholder file in every subfolder of {cmd:folder()}, whether empty or not{p_end}
{synopt :{cmd:skip}}If option {cmd:all} is used and a folder has a file with same name as placeholder file, then nothing is done{p_end}
{synopt :{cmd:replace}}If option {cmd:all} is used and a folder has a file with same name as placeholder file, then the file is overwritten{p_end}
{synoptline}

{marker desc}
{title:Description}

{pstd}GitHub does not sync empty folders, or folder that only contain ignored files.
	However, it is common in research projects
	that a folder structure is added to the GitHub repository at the beginning of a project. At
	the time the folder structure is added to the repository, several folders might still
	be empty and GitHub will not sync them, meaning that they will not be available to the full
	team. {cmd:iegitaddmd} is a Stata adaptation of {it:Solution B} in {browse "http://bytefreaks.net/gnulinux/bash/how-to-add-automatically-all-empty-folders-in-git-repository" :this post}.

{pstd}{cmd:iegitaddmd} creates a placeholder file in all empty subfolders of
	the folder specified in {cmd:folder()}. The default file used if no file is specified
	using option {cmd:file()} is called README.md, which is a name and format recognized by
	github.com so that the content of the file is displayed if the folder with that file is
	browsed on GitHub.com. The placeholder file may be removed as
	soon as actual files have been added to the folder. Alternatively, if the option
	{cmd:all} is used, {cmd:iegitaddmd} will create the placeholder file in each subfolder
	of {cmd:folder()}, regardless of them being empty or not.

{marker optslong}
{title:Options}

{phang}{cmd:folder(}{it:file_path}{cmd:)} is the folder path to the project folder where the
	placeholder file will be created in each empty folder.

{phang}{cmd:file(}{it:{help filename}}{cmd:)} allows the user to specify a file saved on
	the computer to be used as the placeholder file	instead of the default README.md file. This
	allows people and organizations to write their own template placeholder files according to their own
	preferences. We recommend that a file of type .md (markdown) and name README.md is used as
	GitHub.com recognizes this name and format and will display the content of the file when someone
	browse to that folder on GitHub.com. But this is not a technical requirement, any file type
	and name can be used as placeholder file.

{phang}{cmd:all} creates the placeholder file in {cmd:folder()} and in every subfolder,
	regardless if they are empty or not. This allows the user to create a placeholder file in every
	folder that can be edited with documentation and instructions to the purpose and usage of
	each subfolder. This option can also be important when .{it:gitignore} is used, as {cmd:iegitaddmd} will not
	create files to subfolders that only has ignored file -- in which case the folder will not
	be synced by GitHub.

{phang}{cmd:skip} and {cmd:replace} tells {cmd:iegitaddmd} what to do if the option {cmd:all} is used
	and if any of the folders contain a file with the same name as the placeholder file the command
	is trying to create. If a file with the same name as the placeholder file exist and neither of these options
	are used, then the command will throw an error. If {cmd:skip} is used, then nothing is done in the
	folder where the file with the same name exists and the command proceeds to the next folder.
	If {cmd:replace} is used then the file with the same name is overwritten with the new placeholder
	file before the command proceeds to the next folder.

{title:Example}

{pstd}{inp:global github_folder "C:\Users\JohnSmith\Documents\GitHub\ProjectA"}{break}{inp:iegitaddmd , folder({it:"$github_folder"})}

{pstd}In the example above, there is a GitHub repository in the folder ProjectA. This
	repository has a folder structure where some folders are still empty but will later
	be populated with files. In order to have all folders, even the empty ones, synced on all
	collaborators' cloned local copies of the repository, the folders need to contain at least
	one file, which is being created by the command.

{title:Acknowledgements}

{pstd}I would like to acknowledge the help in testing and proofreading I received in relation to this command and help file from (in alphabetic order):{p_end}
{pmore}Guadalupe Bedoya{break}Luiza Cardoso de Andrade{break}Mrijan Rimal{break}Benjamin Daniels{break}

{title:Author}

{phang}Kristoffer Bjarkefur, The World Bank, DECIE

{pstd}Please send bug-reports, suggestions and requests for clarifications
		 writing "ietoolkit iegitaddmd" in the subject line to:{break}
		 lcardosodeandrad@worldbank.org

{pstd}You can also see the code, make comments to the code, see the version
		 history of the code, and submit additions or edits to the code through
		 the {browse "https://github.com/worldbank/ietoolkit" :ietoolkit github repository}.

{smcl}
{* 15 Dec 2017}{...}
{hline}
help for {hi:iegitaddmd}
{hline}

{title:Title}

{phang}{cmdab:iegitaddmd} {hline 2} Creates a placeholder README.md file in subfolders of a GitHub repository folder, which is often needed to sync standardized folder structures.

{title:Syntax}

{phang} {cmdab:iegitaddmd} , {cmd:folder(}{it:file_path}{cmd:)} [all]

{marker opts}{...}
{synoptset 18}{...}
{synopthdr:options}
{synoptline}
{synopt :{cmd:folder(}{it:file_path}{cmd:)}}Specifies the folder path to the repository folder{p_end}
{synopt :{cmd:all}}Creates one {it:README.md} file in every subfolder of {cmd:folder()}, whether empty or not{p_end}
{synoptline}

{marker desc}
{title:Description}

{pstd}GitHub does not sync empty folders, or folder that only contain ignored files.
	However, it is common in research projects
	that a folder structure is added to the GitHub repository at the beginning of a project. At
	the time the folder structure is added to the repository, several folders might still
	be empty and GitHub will not sync them, meaning that they will not be available to the full
	team. {cmd:iegitaddmd} is a Stata adaptation of {it:Solution B} in {browse "http://bytefreaks.net/gnulinux/bash/how-to-add-automatically-all-empty-folders-in-git-repository" :this post}.

{pstd}{cmd:iegitaddmd} creates a placeholder README.md file in all empty subfolders of
	the folder specified in {cmd:folder()}. The placeholder file may be removed as
	soon as actual files have been added to the folder. Alternatively, if the option
	{cmd:all} is used, {cmd:iegitaddmd} will create one README.md file in each subfolder
	of {cmd:folder()}, regardless of them being empty or not.

{marker optslong}
{title:Options}

{phang}{cmd:folder(}{it:file_path}{cmd:)} is the folder path to the repository with empty folders where the file {it:README.md} will be created in each empty folder.

{phang}{cmd:all} creates one {it:README.md} in every subfolder of {cmd:folder()}, whether empty or not. This allows the user to edit the {it:README.md} files and
	add instructions on the purpose and usage of each subfolder. It is also important when a .{it:gitignore} is used, as {cmd:iegitaddmd} will not
	create add files to subfolders that are not empty, but only contain ignored files -- in which case the subfolder will not be synced.

{title:Example}

{pstd}{inp:global github_folder "C:\Users\JohnSmith\Documents\GitHub\ProjectA"}{break}{inp:iegitaddmd , folder({it:"$github_folder"})}

{pstd}In the example above, there is a GitHub repository in the folder ProjectA. This
	repository has a folder structure where some folders are still empty but will later
	be populated with files. In order to have all folders, even the empty ones, synced on all
	collaborators' cloned local copies of the repository, the folders need to contain at least
	one file, which is being created by the command.

{title:Acknowledgements}

{pstd}I would like to acknowledge the help in testing and proofreading I received in relation to this command and help file from (in alphabetic order):{p_end}
{pmore}Guadalupe Bedoya{break}Luiza Cardoso de Andrade{break}Mrijan Rimal{break}Benjamin Daniels{break}

{title:Author}

{phang}Kristoffer Bjarkefur, The World Bank, DECIE

{pstd}Please send bug-reports, suggestions and requests for clarifications
		 writing "ietoolkit iegitaddmd" in the subject line to:{break}
		 lcardosodeandrad@worldbank.org

{pstd}You can also see the code, make comments to the code, see the version
		 history of the code, and submit additions or edits to the code through
		 the {browse "https://github.com/worldbank/ietoolkit" :ietoolkit github repository}.


Update evaluation/all.do

Update evaluation/all.do

for f in case-definitions/*; do
    redo-ifchange $(basename $f .yaml).concepts.json
done

#!/bin/sh
for f in case-definitions/*; do
    echo $f
    redo-ifchange $(basename $f .yaml).index.json
done


iefolder runfile edit

iefolder runfile edit

	*Set standardized settings
	ieboilstart, v(11.1)
	`r(version)'

	*Create filepaths
	*global ietoolkit "C:\Users\wb503680\Box Sync\DIME WOrk\GitFolder\ietoolkit"
	global ietoolkit "C:\Users\wb462869\Box Sync\Stata\Stata work\Commands\ietoolkit"
	global testOuput "$ietoolkit/test/ieimpgraphOutput"

	*Set cd folder to catch and cd leakage
	cd "$ietoolkit\test\cdjunk"

	*Load the command
	do "$ietoolkit\ieimpgraph.do"

	**********************
	*Graph 1 - single treatment

	*open an example data set and create a mock dummy
	sysuse auto, clear
	gen Treatment = (weight >= 3019.5)

	*Run the impact regression
	reg price  Treatment
	*Create the graph
	ieimpgraph Treatment, title("Treatment effect on price") save("$testOuput/Graph1.gph")

	**********************
	*Graph 2 - multiple treatment

	*open an example data set and create mutaually exclusive mock dummies
	sysuse auto, clear
	tab rep78, gen(d_)

	*Run the impact regression
	reg price  d_2 d_3 d_4 d_5 weight length gear_ratio
	*Create the graph
	ieimpgraph d_2 d_3 d_4 d_5, title("Treatment effect on price") save("$testOuput/Graph2.gph")

	*Set standardized settings
	ieboilstart, v(11.1)
	`r(version)'

	*Create filepaths
	*global ietoolkit "C:\Users\wb503680\Box Sync\DIME WOrk\GitFolder\ietoolkit"
	global ietoolkit "C:\Users\wb462869\Box Sync\Stata\Stata work\Commands\ietoolkit"
	global testOuput "$ietoolkit/test/ieimpgraphOutput"

	*Set cd folder to catch and cd leakage
	cd "$ietoolkit\test\cdjunk"

	*Load the command
	do "$ietoolkit\ieimpgraph2.do"

	**********************
	*Graph 1 - single treatment

	*open an example data set and create a mock dummy
	sysuse auto, clear
	gen Treatment = (weight >= 3019.5)

	*Run the impact regression
	reg price  Treatment
	*Create the graph
	ieimpgraph Treatment, title("Treatment effect on price") save("$testOuput/Graph1.gph")

	**********************
	*Graph 2 - multiple treatment

	*open an example data set and create mutaually exclusive mock dummies
	sysuse auto, clear
	tab rep78, gen(d_)

	*Run the impact regression
	reg price  d_2 d_3 d_4 d_5 weight length gear_ratio
	*Create the graph
	ieimpgraph d_2 d_3 d_4 d_5, title("Treatment effect on price") save("$testOuput/Graph2.gph")


Fix use of config.sh in example

Fix use of config.sh in example

redo-ifchange config.sh
. ./config.sh
exec >$3
cat <<-EOF
	redo-ifchange \$1.c
	gcc $CFLAGS -MD -MF \$3.deps.tmp -o \$3 -c \$1.c
	DEPS=\$(sed -e "s/^\$3://" -e 's/\\\\//g' <\$3.deps.tmp)
	rm -f \$3.deps.tmp
	redo-ifchange \$DEPS
EOF
chmod +x $3

redo-ifchange config.sh
. ./config.sh
exec >$3
cat <<-EOF
	redo-ifchange \$1.c
	gcc -MD -MF \$3.deps.tmp -o \$3 -c \$1.c
	DEPS=\$(sed -e "s/^\$3://" -e 's/\\\\//g' <\$3.deps.tmp)
	rm -f \$3.deps.tmp
	redo-ifchange \$DEPS
EOF
chmod +x $3


Don't display command.

Don't display command.

* Call OpenIRT
*
capture program drop openirt
program  openirt
	preserve
	set more off
	set eolchar unix
	local seed = round(runiform()*10000000)
	syntax, id(varname) item_prefix(name) save_item_parameters(string) save_trait_parameters(string) ///
		[samplesize(integer 2000) burnin(integer 1000) theta(varname) model(string) ///
		fixed_item_file(string)]

	isid `id'
	aorder `item_prefix'*

	* List selected items
	display as text "Selected items: " _continue
	foreach v of varlist `item_prefix'* {
		local itemid : subinstr local v "`item_prefix'" "", all
		local item_list `item_list' `itemid'
	}
	display "`item_list'"

	* Write out response data
	display as text "Setting up response data..."
	tempfile response_file
	tempvar group

	if("`theta'" == "") {
		tempvar theta
		qui gen `theta' = -9999
	}
	else {
		qui count if theta != .
		display as text "Number of fixed theta: `r(N)'"
	}

	* check item responses
	foreach v of varlist `item_prefix'* {
		qui sum `v'
		if(r(min)<0 | r(max) > 1) {
			display as error "`v': Response data must be coded 0/1."
			exit
		}
		if(r(mean)==0) {
			display as error "Warning: `v' is 100 percent 0, consider dropping item."
		}
		if(r(mean)==1) {
			display as error "Warning: `v' is 100 percent 1, consider dropping item."
		}
	}
	qui recode _all (.=-9999)
	qui gen `group' = 1
	qui outsheet id `group' `theta' `item_prefix'* using `response_file', replace delim(" ") noquote nolabel nonames
	* type `response_file'
	if("`c(os)'"=="Windows") {
		qui findfile DOS2UNIX.EXE
		! `r(fn)' `response_file'
	}
	* Write out parameter data
	display as text "Setting up parameter data..."
	clear
	local num_items : list sizeof item_list
	qui set obs `num_items'
	qui gen id = .
	local i = 1
	foreach	id of local item_list {
		qui replace id = `id' in `i'
		local ++i
	}
	if("`model'" == "2PL") {
		gen type = 1
		display as text "Default model: Two parameter logistic (2PL)"
	}
	else {
		gen type = 2
		display as text "Default model: Three parameter logistic (3PL)"
	}
	gen numcat = 2
	gen a = -9999
	gen b = -9999
	gen c = -9999
	gen d1 = -9999
	gen d2 = -9999
	gen d3 = -9999
	gen d4 = -9999
	tempfile item_file
	sort id
	qui save `item_file', replace

	if("`fixed_item_file'" != "") {
		cap use `fixed_item_file', clear
		if(_rc != 0) {
			display as error "Could not open fixed item parameter file: `fixed_item_file'."
			exit
		}
		confirm variable id
		confirm variable type
		confirm variable a
		confirm variable b
		confirm variable c
		keep id type a b c
		sort id
		qui merge id using `item_file', update
		qui count if _m==1
		if(`r(N)' > 0) {
			display as error "Fixed item ids do not match response item ids."
			exit
		}
	}
	qui compress
	sort id
	isid id
	qui outsheet id type numcat a b c d1 d2 d3 d4 using `item_file', replace delim(" ") noquote nolabel nonames
	if("`c(os)'"=="Windows") {
		qui findfile DOS2UNIX.EXE
		! `r(fn)' `item_file'
	}
	* Run estimation routine from shell
	qui findfile openirt.exe
	local execfile `r(fn)'
	local execfile : subinstr local execfile " " "\ ", all
	!chmod +x `execfile'

	qui findfile openirt.ini

	*local inifile : subinstr local inifile " " "\ ", all
	*display "`inifile'"
	tempfile inifile
	qui copy "`r(fn)'" `inifile', replace
	tempfile testout
	tempfile responseout
	capture erase `reponseout'
	capture erase `testout'
	local a `execfile' --config-file=`inifile' --test-file=`item_file' --response-file=`response_file' ///
    --test-outfile=`testout' --response-outfile=`responseout' ///
    --sample-size=`samplesize' --burnin=`burnin' --thin=1 --random-seed=`seed'
	* display "`a'"
	! `a'

	cap confirm file `responseout'
	if(_rc == 601) {
		display as error "OpenIRT could not estimate your model.  Please check the specified options and try again."
		exit
	}

	* Test parameters
	qui insheet using `testout', clear
	lab var id "Item ID"
	lab var type "Item Type"
	lab define itemtype 1"2PL" 2"3PL"
	lab val type itemtype
	lab var a_eap "Item discrimination (EAP)"
	lab var b_eap "Item difficulty (EAP)"
	lab var c_eap "Item difficulty (EAP)"
	forvalues i = 1/5 {
		lab var a_pv`i' "Item discrimination (Plausible value `i')"
		lab var b_pv`i' "Item difficulty (Plausible value `i')"
		lab var c_pv`i' "Item guessing (Plausible value `i')"
	}
	qui compress
	qui recode _all (-9999=.)
	sort id
	display as text _newline "Saving item parameters..."
	save `save_item_parameters', replace

	qui insheet using `responseout', clear
	lab var id "Respondent ID"
	drop group
	lab var theta_eap "Theta (EAP)"
	forvalues i = 1/5 {
		lab var theta_pv`i' "Theta (Plausible value `i')"
	}
	lab var theta_mle "Theta (MLE)"
	lab var theta_mle_se "Theta standard error (MLE)"

	* some standard errors may be nan if likelihood very flat.  replace with missing
	qui destring _all, force replace
	qui recode _all (-9999=.)

	display as text _newline "Saving traits..."
	qui compress
	sort id
	save `save_trait_parameters', replace
	restore
	set more on
end

* net install /Users/tristanz/Data/OpenIRT/Stata/openirt, force
* sysuse naep_children, clear
* openirt, id(id) item_prefix(item) save_item_parameters(items) save_trait_parameters(traits)

* Call OpenIRT
*
capture program drop openirt
program  openirt
	preserve
	set more off
	set eolchar unix
	local seed = round(runiform()*10000000)
	syntax, id(varname) item_prefix(name) save_item_parameters(string) save_trait_parameters(string) ///
		[samplesize(integer 2000) burnin(integer 1000) theta(varname) model(string) ///
		fixed_item_file(string)]

	isid `id'
	aorder `item_prefix'*

	* List selected items
	display as text "Selected items: " _continue
	foreach v of varlist `item_prefix'* {
		local itemid : subinstr local v "`item_prefix'" "", all
		local item_list `item_list' `itemid'
	}
	display "`item_list'"

	* Write out response data
	display as text "Setting up response data..."
	tempfile response_file
	tempvar group

	if("`theta'" == "") {
		tempvar theta
		qui gen `theta' = -9999
	}
	else {
		qui count if theta != .
		display as text "Number of fixed theta: `r(N)'"
	}

	* check item responses
	foreach v of varlist `item_prefix'* {
		qui sum `v'
		if(r(min)<0 | r(max) > 1) {
			display as error "`v': Response data must be coded 0/1."
			exit
		}
		if(r(mean)==0) {
			display as error "Warning: `v' is 100 percent 0, consider dropping item."
		}
		if(r(mean)==1) {
			display as error "Warning: `v' is 100 percent 1, consider dropping item."
		}
	}
	qui recode _all (.=-9999)
	qui gen `group' = 1
	qui outsheet id `group' `theta' `item_prefix'* using `response_file', replace delim(" ") noquote nolabel nonames
	* type `response_file'
	if("`c(os)'"=="Windows") {
		qui findfile DOS2UNIX.EXE
		! `r(fn)' `response_file'
	}
	* Write out parameter data
	display as text "Setting up parameter data..."
	clear
	local num_items : list sizeof item_list
	qui set obs `num_items'
	qui gen id = .
	local i = 1
	foreach	id of local item_list {
		qui replace id = `id' in `i'
		local ++i
	}
	if("`model'" == "2PL") {
		gen type = 1
		display as text "Default model: Two parameter logistic (2PL)"
	}
	else {
		gen type = 2
		display as text "Default model: Three parameter logistic (3PL)"
	}
	gen numcat = 2
	gen a = -9999
	gen b = -9999
	gen c = -9999
	gen d1 = -9999
	gen d2 = -9999
	gen d3 = -9999
	gen d4 = -9999
	tempfile item_file
	sort id
	qui save `item_file', replace

	if("`fixed_item_file'" != "") {
		cap use `fixed_item_file', clear
		if(_rc != 0) {
			display as error "Could not open fixed item parameter file: `fixed_item_file'."
			exit
		}
		confirm variable id
		confirm variable type
		confirm variable a
		confirm variable b
		confirm variable c
		keep id type a b c
		sort id
		qui merge id using `item_file', update
		qui count if _m==1
		if(`r(N)' > 0) {
			display as error "Fixed item ids do not match response item ids."
			exit
		}
	}
	qui compress
	sort id
	isid id
	qui outsheet id type numcat a b c d1 d2 d3 d4 using `item_file', replace delim(" ") noquote nolabel nonames
	if("`c(os)'"=="Windows") {
		qui findfile DOS2UNIX.EXE
		! `r(fn)' `item_file'
	}
	* Run estimation routine from shell
	qui findfile openirt.exe
	local execfile `r(fn)'
	local execfile : subinstr local execfile " " "\ ", all
	!chmod +x `execfile'

	qui findfile openirt.ini

	*local inifile : subinstr local inifile " " "\ ", all
	*display "`inifile'"
	tempfile inifile
	qui copy "`r(fn)'" `inifile', replace
	tempfile testout
	tempfile responseout
	capture erase `reponseout'
	capture erase `testout'
	local a `execfile' --config-file=`inifile' --test-file=`item_file' --response-file=`response_file' ///
    --test-outfile=`testout' --response-outfile=`responseout' ///
    --sample-size=`samplesize' --burnin=`burnin' --thin=1 --random-seed=`seed'
	display "`a'"
	! `a'

	cap confirm file `responseout'
	if(_rc == 601) {
		display as error "OpenIRT could not estimate your model.  Please check the specified options and try again."
		exit
	}

	* Test parameters
	qui insheet using `testout', clear
	lab var id "Item ID"
	lab var type "Item Type"
	lab define itemtype 1"2PL" 2"3PL"
	lab val type itemtype
	lab var a_eap "Item discrimination (EAP)"
	lab var b_eap "Item difficulty (EAP)"
	lab var c_eap "Item difficulty (EAP)"
	forvalues i = 1/5 {
		lab var a_pv`i' "Item discrimination (Plausible value `i')"
		lab var b_pv`i' "Item difficulty (Plausible value `i')"
		lab var c_pv`i' "Item guessing (Plausible value `i')"
	}
	qui compress
	qui recode _all (-9999=.)
	sort id
	display as text _newline "Saving item parameters..."
	save `save_item_parameters', replace

	qui insheet using `responseout', clear
	lab var id "Respondent ID"
	drop group
	lab var theta_eap "Theta (EAP)"
	forvalues i = 1/5 {
		lab var theta_pv`i' "Theta (Plausible value `i')"
	}
	lab var theta_mle "Theta (MLE)"
	lab var theta_mle_se "Theta standard error (MLE)"

	* some standard errors may be nan if likelihood very flat.  replace with missing
	qui destring _all, force replace
	qui recode _all (-9999=.)

	display as text _newline "Saving traits..."
	qui compress
	sort id
	save `save_trait_parameters', replace
	restore
	set more on
end

* net install /Users/tristanz/Data/OpenIRT/Stata/openirt, force
* sysuse naep_children, clear
* openirt, id(id) item_prefix(item) save_item_parameters(items) save_trait_parameters(traits)


use csh-friendly stderr redirection

use csh-friendly stderr redirection

This fixes parallel for (t)csh users. This syntax is compatible with
at least bash, zsh and tcsh, which should cover the majority of
Linux and MacOS users.

Users of ksh and plainer sh varieties will need to override their
shell to one of the above prior to running Stata, e.g.
    env SHELL=/bin/bash xstata ...

// Succeeds if Process ID has finished (returns ret_code if can get) and fails otherwise.
// Maybe eventually allow waiting for a fixed period of time
//  (but have to be careful to avoid deadlocks, and maybe waiting in Stata is better
//   because you can handle breaks better?)
/* Usage:
cap noi procwait <PID>
//cap noi procwait <PID>, connection(ssh node2)
if(!_rc) <something if finished>
*/
program procwait, rclass
	version 11.0
	syntax anything(everything) [, connection(string)]

	if (c(os) == "Windows") {
		plugin call procwait_plug, `"`anything'"'
		if "`ret_code'"!="" return local ret_code "`ret_code'"
	}
	else {
		//kill -0 is the POSIX way of checking
		tempfile kill_exit_code
		shell `connection' kill -0 `anything' >&/dev/null; echo \$? > "`kill_exit_code'"
		//kill exits with 0 if exists, 1 otherwise

		tempname fh
		file open `fh' using `"`kill_exit_code'"', read text
		file read `fh' not_active
		file close `fh'

		if !`not_active' error 1
	}
end

cap program procwait_plug, plugin using("procwait.plugin")

// Succeeds if Process ID has finished (returns ret_code if can get) and fails otherwise.
// Maybe eventually allow waiting for a fixed period of time
//  (but have to be careful to avoid deadlocks, and maybe waiting in Stata is better
//   because you can handle breaks better?)
/* Usage:
cap noi procwait <PID>
//cap noi procwait <PID>, connection(ssh node2)
if(!_rc) <something if finished>
*/
program procwait, rclass
	version 11.0
	syntax anything(everything) [, connection(string)]

	if (c(os) == "Windows") {
		plugin call procwait_plug, `"`anything'"'
		if "`ret_code'"!="" return local ret_code "`ret_code'"
	}
	else {
		//kill -0 is the POSIX way of checking
		tempfile kill_exit_code
		shell `connection' kill -0 `anything' 2> /dev/null; echo \$? > "`kill_exit_code'"
		//kill exits with 0 if exists, 1 otherwise

		tempname fh
		file open `fh' using `"`kill_exit_code'"', read text
		file read `fh' not_active
		file close `fh'

		if !`not_active' error 1
	}
end

cap program procwait_plug, plugin using("procwait.plugin")


move matrix documentation to one place

move matrix documentation to one place

cap program drop 	ieddtable
	program define	ieddtable

	syntax varlist, ///
					///
		t(varname numeric) tmt(varname numeric) 			///
		[ 													///
		COVARiates(varlist numeric)							///
		STARLevels(numlist descending min=3 max=3 >0 <1)	///
		STARSNOadd											///
		firstdiff											///
		rowlabtype(string) 									///
		rowlabtext(string)									///
		errortype(string)									///
		]

	/*************

		Input handling

	*************/

	*LABELS
	*Test and prepare the row lables and test how long the longest label is.
	prepRowLabels `varlist', rowlabtype("`rowlabtype'") rowlabtext("`rowlabtext'")
	local rowlabels "`r(rowlabels)'"
	local labmaxlen "`r(rowlabel_maxlen)'"

	*ERRORTYPES
	*If error type is used test that it is only one word and that word is an allowed error type
	if "`errortype'" != "" {
		local errortype = lower("`errortype'")
		if (`: word count `errortype'' != 1) | !inlist("`errortype'", "sd", "se") {
			noi di as error "Value in option errortype(`errortype') can only one out of the following words; se, sd."
			error 198
		}
	}
	else {
		*No errortype specified use default which is standard errors
		local errortype "se"
	}

	/*************

		Initiate the result matrix

	*************/

	*Creates the template for the result matrix in a subfunction
	templateResultMatrix
	mat startRow = r(startRow)
	local colnames = "`r(colnames)'"

	*Remove this when ready for production
	//noi di "Start row to see headers, remove for production"
	//matlist startRow // See the default row with its column names

	*Initiate the result matrix with a place holder row that will not be used for anything. Matrices cannot be initiated empty
	mat ddtab_resultMap = startRow
	mat rownames ddtab_resultMap = placeholder

	/*************

		Loop over all variables and prepare the data

	*************/

	foreach var of local varlist {

		//noi di "Variable `var'"

		*Each variable is a row in the result matrix
		mat `var' = startRow
		mat rownames `var' = `var'
		mat colnames `var' = `colnames'

		*Local that keeps track of which column to fill
		local colindex 0

		tempvar regsample

		/*************

			Calculating 2nd difference and restict the sample
			for this output var

		*************/

		*Run the regression to get the double difference
		qui reg `var' `tmt'#`t' `covariates'
		mat resTable = r(table)

		//noi di "Rest table 2nd diff: reg `var' `tmt'#`t' `covariates'"
		//matlist resTable

		**This is why this is done first. All other calculations
		* for this outcome var should be restricted to this sample.
		gen `regsample' = e(sample)
		local N `e(N)'

		**Test that the dummy vars creates for valid groups for each
		* combination of time and treat in the sample used for this outcome var
		testDDdums `t' `tmt' `regsample' `var'  //comment in when this is made into a command

		*Get the second differnce
		local ++colindex
		mat `var'[1,`colindex'] =  el(resTable,1,4)

		*Get the standard error of second difference
		local ++colindex
		convertErrs el(resTable,2,4) `N' "`errortype'"
		mat `var'[1,`colindex'] =  `r(converted_error)'

		*Get the number of stars using sub-command countStars
		local ++colindex
		local pvalue = el(resTable,4,4)
		countStars `pvalue' `starlevels'
		mat `var'[1,`colindex'] = `r(stars)'

		*Get the N of second difference regression
		local ++colindex
		mat `var'[1,`colindex'] = `N'

		/*************

			Calculating 1st differences

		*************/
		forvalues tmt01 = 0/1 {

			*Regress time against the outcome var one tmt group at the time
			qui reg `var' `t' `covariates' if `tmt' == `tmt01' & `regsample' == 1
			mat resTable = r(table)

			//noi di "Rest table 1st diff: reg `var' `t' `covariates' if `tmt' == `tmt01' & `regsample' == 1"
			//matlist resTable

			//Get the 1st diff
			local ++colindex
			mat `var'[1,`colindex'] =  el(resTable,1,1)

			*Get the standard error of 1st diff
			local ++colindex
			convertErrs el(resTable,2,1) `e(N)' "`errortype'"
			mat `var'[1,`colindex'] =  `r(converted_error)'

			*Get the number of stars using sub-command countStars
			local ++colindex
			local pvalue = el(resTable,4,4)
			countStars `pvalue' `starlevels'
			mat `var'[1,`colindex'] = `r(stars)'

			*Get the N of first difference regression
			local ++colindex
			mat `var'[1,`colindex'] =  `e(N)'

		}

		/*************

			Calculating standard means for all groups

		*************/
		forvalues tmt01 = 0/1 {
			forvalues t01 = 0/1 {

				*Summary stats on this group
				qui mean `var' if `tmt' == `t01' & `t' == `tmt01' & `regsample' == 1

				mat resTable = r(table)

				//Get the mean
				local ++colindex
				mat `var'[1,`colindex'] =  el(resTable,1,1)

				*Get the standard error of the mean
				local ++colindex
				convertErrs el(resTable,2,1)  `e(N)' "`errortype'"
				mat `var'[1,`colindex'] =  `r(converted_error)'

				*Get the N of the ordinary means regressions
				local ++colindex
				mat `var'[1,`colindex'] =  `e(N)'
			}
		}

		*Append this row to the result table
		mat 	ddtab_resultMap = (ddtab_resultMap \ `var')

	}

	*Remove placeholder row
	matrix ddtab_resultMap = ddtab_resultMap[2..., 1...]

	*Show the final matrix will all data needed to start building the output
	//noi di "Matlist with results"
	//matlist ddtab_resultMap

	/*************

		Output table in result window

	*************/

	outputwindow `varlist' , ddtab_resultMap(ddtab_resultMap) labmaxlen(`labmaxlen') rwlbls(`rowlabels') ///
		covariates(`covariates')

	/*************

		Output table in LaTeX

	*************/

	if "`savetex'" != "" {

		//outputtex `varlist' , ddtab_resultMap(ddtab_resultMap) savetex(`savetex')

	}

end

/***************************************
****************************************

	Write sub-commands for input handling

****************************************
***************************************/

**Program to prepare row labels for each outcome var, using variable name,
* variable label or manually entered labels depedning on user input. Variable
* names are used if no input.
cap program drop 	prepRowLabels
	program define	prepRowLabels, rclass

	syntax varlist, [rowlabtype(string) rowlabtext(string)]

	*First test input
	if "`rowlabtype'" == "varlab" | "`rowlabtype'" == "" | "`rowlabtype'" == "varname" {
		//All is good do nothing
	}
	else {
		noi display as error "{phang}Row label type [`rowlabtype'] is not a valid row label type. Enter either {it:varlab} or {it:varname} (the default if not specified is {it:varname}). See option {help ieddtable:rowlabtype} for details."
		error 198
	}

	/*************
		Parse through the manually entered labels
	*************/

	*Create a local with the rowlabel input to be tokenized
	local row_labels_to_tokenize `rowlabtext'

	while "`row_labels_to_tokenize'" != "" {

		*Parsing name and label pair
		gettoken nameAndLabel row_labels_to_tokenize : row_labels_to_tokenize, parse("@@")

		*Splitting name and label
		gettoken name label : nameAndLabel

		/*
			We store two locals, one with varnames and one with label. They are
			stored in the same order, so if we find varname we know we will find
			the corresponding label in the same order in the other local.
		*/

		*** Tests

		*Checking that the variables used in rowlabels() are included in the table
		local name_correct : list name in varlist
		if `name_correct' == 0 {
			noi display as error "{phang}Variable [`name'] listed in rowlabtext(`rowlabtext') is not found among the outcome variables."
			error 111
		}

		*Testing that no label is missing
		if "`label'" == "" {
			noi display as error "{phang}For variable [`name'] listed in rowlabtext(`rowlabels') you have not specified any label. Labels are requried for all variables listed in rowlabels(). Variables omitted from rowlabtext() will be assigned labels according to the rule in rowlabtype(). See also option {help ieddtable:rowlabtext}"
			error 198
		}

		*Storing the name in local to be used to get index of corresponding label in the second local
		local rowLabelNames `"`rowLabelNames' "`name'" "'

		*Removing leading or trailing spaces and store label with the same index as in the name local above
		local label = trim("`label'")
		local rowLabelLabels `"`rowLabelLabels' "`label'" "'

		*Parse char is not removed by gettoken, so remove it and start over
		local row_labels_to_tokenize = subinstr("`row_labels_to_tokenize'" ,"@@","",1)
	}

	**Set up locals to be returned. One with row labels without varname but in
	* final order, and one local with length of longest label
	local preppedRowLabels
	local maxLen 0

	*Loop over all varaibles and prepare the labels
	foreach var of local varlist {

		**Get the index of this variable in manually entered labels. Index is
		* zero if no manually entered label for this variable
		local rowLabPos : list posof "`var'" in rowLabelNames

		if `rowLabPos' == 0 {

			*No label was manually entered for this variable, use the other rules
			if "`rowlabtype'" == "varlab" {
				local this_label : variable label `var'		//Use variable label
			}
			else {

				local this_label `var'						//Default, use varname

			}
		}
		else {
			*Getting the manually defined label corresponding to this variable
			local this_label : word `rowLabPos' of `rowLabelLabels'
		}

		*Update the two locals that will be returned
		local preppedRowLabels "`preppedRowLabels' @@`this_label'"	//Prepare the final list
		local maxLen = max(strlen("`this_label'"),`maxLen')	//Store the length on the longest label, used on some outputs
	}

	*Return the locals
	return local rowlabels `preppedRowLabels'
	return local rowlabel_maxlen `maxLen'

end

cap program drop 	testDDdums
	program define	testDDdums

		args time treat samplevar outputvar

		*Test taht the dummies are dummies
		foreach dummy in `time' `treat' {

			cap assert inlist(`dummy',1,0) | missing(`dummy') | `samplevar' == 0
			if _rc {
				noi di as error "{phang}In the difference in differnce regression for outputvar [`outputvar'], the dummy [`dummy'] is not a real dummy as it has values differnt from 1 and 0. See tab below. The tab is restricted to the observations that are not excluded due to missing values etc. in the difference in differnce regression.{p_end}""
				noi tab `dummy' if `samplevar' == 1, missing
				error 480
			}
		}

		**Test that for only two of three dummies there are observations
		* that has only that dummy. I.e. the two that is not the
		* interaction. If the interaction is 1, all three shluld be 1.

		*Test that there is some observations in each group
		forvalues tmt01 = 0/1 {
			forvalues t01 = 0/1 {

				*Summary stats on this group
				qui count if `treat' == `t01' & `time' == `tmt01' & `samplevar' == 1

				//There got to be more than 1 observation in each group for the regression to make sense
				if `r(N)' < 2 {
					noi di as error "{phang}In the difference in differnce regression for outputvar [`outputvar'] there were not enough observations in each combination of treatment [`treat'] and time [`time']. The tab below is restricted to the observations that are not excluded due to missing values etc. in the difference in differnce regression.{p_end}""
					noi tab `treat' `time' if `samplevar' == 1, missing
					error 480

				}
			}
		}

	end

/***************************************
****************************************

	Write sub-commands for stats

****************************************
***************************************/

*Sets up template for the result matrix
cap program drop 	templateResultMatrix
	program define	templateResultMatrix, rclass

	local 2ndDiff_cols 			2D 2D_err 2D_stars 2D_N

	local 1stDiff_C_cols		1DC 1DC_err 1DC_stars 1DC_N
	local 1stDiff_T_cols		1DT 1DT_err 1DT_stars 1DT_N

	local basicMean_C0_cols		C0_mean C0_err C0_N
	local basicMean_T0_cols		T0_mean T0_err T0_N
	local basicMean_C1_cols		C1_mean C1_err C1_N
	local basicMean_T1_cols		T1_mean T1_err T1_N

	local colnames `2ndDiff_cols' `1stDiff_C_cols' `1stDiff_T_cols' `basicMean_C0_cols' `basicMean_T0_cols' `basicMean_C1_cols' `basicMean_T0_cols'

	*Define default row here. The results for each var will be one row that starts with all missing vlaues
	mat startRow = (.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.)
	mat colnames startRow = `colnames'

	return matrix startRow startRow
	return local colnames `colnames'

end

*Take the significance levels and count number of starts
cap program drop 	countStars
	program define	countStars, rclass

	args pvalue star1 star2 star3

	*Test if custom star levels are used
	if "`star1'" == "" {

		*Set default levels for 1, 2 and 3 stars
		local star1 .1
		local star2 .05
		local star3 .01
	}

	local stars 0
	foreach star_p_level in `star1' `star2' `star3' {

		if `pvalue' < `star_p_level' local ++stars
	}

	return local stars `stars'

end

*Take the significance levels and count number of starts
cap program drop 	convertErrs
	program define	convertErrs, rclass

	args se_err N errortype

	*Test if custom star levels are used
	if "`errortype'" == "se" {
		local err `se_err'
	}
	else if "`errortype'" == "sd" {
		local err = `se_err' * sqrt(`N')
	}

	return local converted_error `err'

end

/***************************************
****************************************

	Write sub-commands for outputs

****************************************
***************************************/

	******
	*Then the result matrix can be passed into subcommands that output in either Excel, LaTeX or in the main window.

	*The results can be accessed like this, which makes the sub-commands less sensitive to changing column order in the section above.
	//mat A = ddtab_resultMap[3, "C0_Mean"] // returns a 1x1 matrix with the baseline mean for the countrol grop for the thrid outcome var
	//local a = el(A,1,1)				// returns the value

	//matlist A
	//di `a'

	/*
		Column name dictionary
			Differences:
			- 2D : Second differnce coefficient (t*tmt == 1)
			- 1DC : First  differnce coefficient control (t == 0)
			- 1DT : First  differnce coefficient treatment (tmt == 0)

			for each coefficent these stats are also provided:
				- _err : Second differnce errors (type of errors is set in command errortype)
				- _stars :  Second Differene - The number of significance stars (sig level set in command)
				- _N : Second Difference - Number of observtions in the regression

			Group means:
			- C0 - Control time 0
			- T0 - Treatment time 0
			- C1 - Control time 1
			- T1 - Treatment time 1

			for each group these stats are also provided:
				- _mean : the mean of the group
				- _err : the error in the mean (type of errors is set in command errortype)

	*/

	cap program drop 	outputwindow
		program define	outputwindow

		/*
		Todo:
			note on stars
			Add errors on second line

		*/

		syntax varlist , ddtab_resultMap(name) labmaxlen(numlist) rwlbls(string) [covariates(string)]

		local numVars = `:word count `varlist''

		local statlist 2D 1DT 1DC C0_mean T0_mean
		local diformat = "%9.2f"

		local first_hhline = 2 + `labmaxlen'
		local first_col = 4 + `labmaxlen'

		local bsln_space = 2
		local diff_space = 3

		local bsln_width = ((`bsln_space' * 2) + 8)
		local diff_width = ((`diff_space' * 2) + 10)

		local ctrl_hline = `bsln_width' + 1 + `diff_width'
		local ctrl_space = (`ctrl_hline' - 7) / 2

		local tmt_hline = `bsln_width' + 1 + `diff_width'
		local tmt_space = (`tmt_hline' - 9) / 2

		noi di as text "{c TLC}{hline `first_hhline'}{c TT}{hline `ctrl_hline'}{c TT}{hline `tmt_hline'}{c TT}{hline 17}{c TRC}"
		noi di as text "{c |}{col `first_col'}{c |}{dup `ctrl_space': }Control{dup `ctrl_space': }{c |}{dup `tmt_space': }Treatment{dup `tmt_space': }{c |}  Difference-in  {c |}"
		noi di as text "{c |}{col 3}Variable{col `first_col'}{c |}{dup `bsln_space': }Baseline{dup `bsln_space': }{c |}{dup `diff_space': }Difference{dup `diff_space': }{c |}{dup `bsln_space': }Baseline{dup `bsln_space': }{c |}{dup `diff_space': }Difference{dup `diff_space': }{c |}   -difference   {c |}"
		noi di as text "{c LT}{hline `first_hhline'}{c +}{hline `bsln_width'}{c +}{hline `diff_width'}{c +}{hline `bsln_width'}{c +}{hline `diff_width'}{c +}{hline 17}{c RT}"

		//noi di "diff w : `diff_width'"
		//noi di "bsln w : `bsln_width'"

		forvalues row = 1/`numVars' {

			local rwlbls = trim(subinstr("`rwlbls'", "@@","", 1))
			gettoken label rwlbls : rwlbls, parse("@@")

			foreach stat of local statlist {

				displayformatter , statname("`stat'") row(`row') ddtab_resultMap(ddtab_resultMap) diformat("`diformat'") bslnw(`bsln_width') diffw(`diff_width')
				local `stat' `r(disp_stata)'
				local `stat'_space = `r(disp_pre_space)'

				//noi di "`stat' : ``stat'' : `r(disp_len)' : `r(disp_pre_space)'"
			}

			*Disaplay each variable row at the same time
			noi di as text "{c |} `label'{col `first_col'}{c |}{dup `C0_mean_space': }`C0_mean'{dup `1DC_space': }`1DC'{dup `T0_mean_space': }`T0_mean'{dup `1DT_space': }`1DT'{dup `2D_space': }`2D'"

		}

		noi di as text "{c BLC}{hline `first_hhline'}{c BT}{hline `bsln_width'}{c BT}{hline `diff_width'}{c BT}{hline `bsln_width'}{c BT}{hline `diff_width'}{c BT}{hline 17}{c BRC}"

		if ("`covariates'" != "") {

			noi di as text "  The following variables was included as covariates [`covariates']"

		}

	end

cap program drop 	displayformatter
	program define	displayformatter, rclass

	syntax , statname(string) row(numlist) ddtab_resultMap(name) diformat(string)  bslnw(numlist) diffw(numlist)

		mat temp = ddtab_resultMap[`row', "`statname'"]
		local `statname' = el(temp,1,1)
		local `statname' 	: display `diformat' ``statname''

		if inlist("`statname'", "2D", "1DT", "1DC") {

			mat starNumMat = ddtab_resultMap[`row', "`statname'_stars"]
			local starNum = el(starNumMat,1,1)

			if `starNum' == 0 local `statname' "``statname''   "
			if `starNum' == 1 local `statname' "``statname''*  "
			if `starNum' == 2 local `statname' "``statname''** "
			if `starNum' == 3 local `statname' "``statname''***"
		}

		local `statname' = ltrim("``statname''")

		local len = strlen("``statname''")
		//noi di "|``statname''|"
		//noi di `len'

		local numSpace 0

		if inlist("`statname'", "C0_mean", "T0_mean") {
			local numSpace = `bslnw' - `len' - 1
		}
		else if inlist("`statname'", "1DT", "1DC") {
			local numSpace = `diffw' - `len' - 1
		}
		else if inlist("`statname'", "2D") {
			local numSpace = 16 - `len'
		}

		if `numSpace' < 0 local numSpace 0

		return local disp_pre_space = `numSpace'
		return local disp_len = `len'
		return local disp_stata "``statname'' {c |}"

	end

cap program drop 	ieddtable
	program define	ieddtable

	syntax varlist, ///
					///
		t(varname numeric) tmt(varname numeric) 			///
		[ 													///
		COVARiates(varlist numeric)							///
		STARLevels(numlist descending min=3 max=3 >0 <1)	///
		STARSNOadd											///
		firstdiff											///
		rowlabtype(string) 									///
		rowlabtext(string)									///
		errortype(string)									///
		]

	/*************

		Input handling

	*************/

	*LABELS
	*Test and prepare the row lables and test how long the longest label is.
	prepRowLabels `varlist', rowlabtype("`rowlabtype'") rowlabtext("`rowlabtext'")
	local rowlabels "`r(rowlabels)'"
	local labmaxlen "`r(rowlabel_maxlen)'"
	return list

	*ERRORTYPES
	*If error type is used test that it is only one word and that word is an allowed error type
	if "`errortype'" != "" {
		local errortype = lower("`errortype'")
		if (`: word count `errortype'' != 1) | !inlist("`errortype'", "sd", "se") {
			noi di as error "Value in option errortype(`errortype') can only one out of the following words; se, sd."
			error 198
		}
	}
	else {
		*No errortype specified use default which is standard errors
		local errortype "se"
	}

	/*************

		Initiate the result matrix

	*************/

	*Creates the template for the result matrix in a subfunction
	templateResultMatrix
	mat startRow = r(startRow)
	local colnames = "`r(colnames)'"

	*Remove this when ready for production
	//noi di "Start row to see headers, remove for production"
	//matlist startRow // See the default row with its column names

	*Initiate the result matrix with a place holder row that will not be used for anything. Matrices cannot be initiated empty
	mat ddtab_resultMap = startRow
	mat rownames ddtab_resultMap = placeholder

	/*
		Column name dictionary
			Differences:
			- 2D : Second differnce coefficient (t*tmt == 1)
			- 1DC : First  differnce coefficient control (t == 0)
			- 1DT : First  differnce coefficient treatment (tmt == 0)

			for each coefficent these stats are also provided:
				- _err : Second differnce errors (type of errors is set in command errortype)
				- _stars :  Second Differene - The number of significance stars (sig level set in command)
				- _N : Second Difference - Number of observtions in the regression

			Group means:
			- C0 - Control time 0
			- T0 - Treatment time 0
			- C1 - Control time 1
			- T1 - Treatment time 1

			for each group these stats are also provided:
				- _mean : the mean of the group
				- _err : the error in the mean (type of errors is set in command errortype)

	*/

	/*************

		Loop over all variables and prepare the data

	*************/

	foreach var of local varlist {

		noi di "Variable `var'"

		*Each variable is a row in the result matrix
		mat `var' = startRow
		mat rownames `var' = `var'
		mat colnames `var' = `colnames'

		*Local that keeps track of which column to fill
		local colindex 0

		tempvar regsample

		/*************

			Calculating 2nd difference and restict the sample
			for this output var

		*************/

		*Run the regression to get the double difference
		qui reg `var' `tmt'#`t' `covariates'
		mat resTable = r(table)

		//noi di "Rest table 2nd diff: reg `var' `tmt'#`t' `covariates'"
		//matlist resTable

		**This is why this is done first. All other calculations
		* for this outcome var should be restricted to this sample.
		gen `regsample' = e(sample)
		local N `e(N)'

		**Test that the dummy vars creates for valid groups for each
		* combination of time and treat in the sample used for this outcome var
		testDDdums `t' `tmt' `regsample' `var'  //comment in when this is made into a command

		*Get the second differnce
		local ++colindex
		mat `var'[1,`colindex'] =  el(resTable,1,4)

		*Get the standard error of second difference
		local ++colindex
		convertErrs el(resTable,2,4) `N' "`errortype'"
		mat `var'[1,`colindex'] =  `r(converted_error)'

		*Get the number of stars using sub-command countStars
		local ++colindex
		local pvalue = el(resTable,4,4)
		countStars `pvalue' `starlevels'
		mat `var'[1,`colindex'] = `r(stars)'

		*Get the N of second difference regression
		local ++colindex
		mat `var'[1,`colindex'] = `N'

		/*************

			Calculating 1st differences

		*************/
		forvalues tmt01 = 0/1 {

			*Regress time against the outcome var one tmt group at the time
			qui reg `var' `t' `covariates' if `tmt' == `tmt01' & `regsample' == 1
			mat resTable = r(table)

			//noi di "Rest table 1st diff: reg `var' `t' `covariates' if `tmt' == `tmt01' & `regsample' == 1"
			//matlist resTable

			//Get the 1st diff
			local ++colindex
			mat `var'[1,`colindex'] =  el(resTable,1,1)

			*Get the standard error of 1st diff
			local ++colindex
			convertErrs el(resTable,2,1) `e(N)' "`errortype'"
			mat `var'[1,`colindex'] =  `r(converted_error)'

			*Get the number of stars using sub-command countStars
			local ++colindex
			local pvalue = el(resTable,4,4)
			countStars `pvalue' `starlevels'
			mat `var'[1,`colindex'] = `r(stars)'

			*Get the N of first difference regression
			local ++colindex
			mat `var'[1,`colindex'] =  `e(N)'

		}

		/*************

			Calculating standard means for all groups

		*************/
		forvalues tmt01 = 0/1 {
			forvalues t01 = 0/1 {

				*Summary stats on this group
				qui mean `var' if `tmt' == `t01' & `t' == `tmt01' & `regsample' == 1

				mat resTable = r(table)

				//Get the mean
				local ++colindex
				mat `var'[1,`colindex'] =  el(resTable,1,1)

				*Get the standard error of the mean
				local ++colindex
				convertErrs el(resTable,2,1)  `e(N)' "`errortype'"
				mat `var'[1,`colindex'] =  `r(converted_error)'

				*Get the N of the ordinary means regressions
				local ++colindex
				mat `var'[1,`colindex'] =  `e(N)'
			}
		}

		*Append this row to the result table
		mat 	ddtab_resultMap = (ddtab_resultMap \ `var')

	}

	*Remove placeholder row
	matrix ddtab_resultMap = ddtab_resultMap[2..., 1...]

	*Show the final matrix will all data needed to start building the output
	noi di "Matlist with results"
	matlist ddtab_resultMap

	/*************

		Output table in result window

	*************/

	outputwindow `varlist' , ddtab_resultMap(ddtab_resultMap) labmaxlen(`labmaxlen') rwlbls(`rowlabels') ///
		covariates(`covariates')

	/*************

		Output table in LaTeX

	*************/

	if "`savetex'" != "" {

		//outputtex `varlist' , ddtab_resultMap(ddtab_resultMap) savetex(`savetex')

	}

end

/***************************************
****************************************

	Write sub-commands for input handling

****************************************
***************************************/
if 1 {
**Program to prepare row labels for each outcome var, using variable name,
* variable label or manually entered labels depedning on user input. Variable
* names are used if no input.
cap program drop 	prepRowLabels
	program define	prepRowLabels, rclass

	syntax varlist, [rowlabtype(string) rowlabtext(string)]

	*First test input
	if "`rowlabtype'" == "varlab" | "`rowlabtype'" == "" | "`rowlabtype'" == "varname" {
		//All is good do nothing
	}
	else {
		noi display as error "{phang}Row label type [`rowlabtype'] is not a valid row label type. Enter either {it:varlab} or {it:varname} (the default if not specified is {it:varname}). See option {help ieddtable:rowlabtype} for details."
		error 198
	}

	/*************
		Parse through the manually entered labels
	*************/

	*Create a local with the rowlabel input to be tokenized
	local row_labels_to_tokenize `rowlabtext'

	while "`row_labels_to_tokenize'" != "" {

		*Parsing name and label pair
		gettoken nameAndLabel row_labels_to_tokenize : row_labels_to_tokenize, parse("@@")

		*Splitting name and label
		gettoken name label : nameAndLabel

		/*
			We store two locals, one with varnames and one with label. They are
			stored in the same order, so if we find varname we know we will find
			the corresponding label in the same order in the other local.
		*/

		*** Tests

		*Checking that the variables used in rowlabels() are included in the table
		local name_correct : list name in varlist
		if `name_correct' == 0 {
			noi display as error "{phang}Variable [`name'] listed in rowlabtext(`rowlabtext') is not found among the outcome variables."
			error 111
		}

		*Testing that no label is missing
		if "`label'" == "" {
			noi display as error "{phang}For variable [`name'] listed in rowlabtext(`rowlabels') you have not specified any label. Labels are requried for all variables listed in rowlabels(). Variables omitted from rowlabtext() will be assigned labels according to the rule in rowlabtype(). See also option {help ieddtable:rowlabtext}"
			error 198
		}

		*Storing the name in local to be used to get index of corresponding label in the second local
		local rowLabelNames `"`rowLabelNames' "`name'" "'

		*Removing leading or trailing spaces and store label with the same index as in the name local above
		local label = trim("`label'")
		local rowLabelLabels `"`rowLabelLabels' "`label'" "'

		*Parse char is not removed by gettoken, so remove it and start over
		local row_labels_to_tokenize = subinstr("`row_labels_to_tokenize'" ,"@@","",1)
	}

	**Set up locals to be returned. One with row labels without varname but in
	* final order, and one local with length of longest label
	local preppedRowLabels
	local maxLen 0

	*Loop over all varaibles and prepare the labels
	foreach var of local varlist {

		**Get the index of this variable in manually entered labels. Index is
		* zero if no manually entered label for this variable
		local rowLabPos : list posof "`var'" in rowLabelNames

		if `rowLabPos' == 0 {

			*No label was manually entered for this variable, use the other rules
			if "`rowlabtype'" == "varlab" {
				local this_label : variable label `var'		//Use variable label
			}
			else {

				local this_label `var'						//Default, use varname

			}
		}
		else {
			*Getting the manually defined label corresponding to this variable
			local this_label : word `rowLabPos' of `rowLabelLabels'
		}

		*Update the two locals that will be returned
		local preppedRowLabels "`preppedRowLabels' @@`this_label'"	//Prepare the final list
		local maxLen = max(strlen("`this_label'"),`maxLen')	//Store the length on the longest label, used on some outputs
	}

	*Return the locals
	return local rowlabels `preppedRowLabels'
	return local rowlabel_maxlen `maxLen'

end

cap program drop 	testDDdums
	program define	testDDdums

		args time treat samplevar outputvar

		*Test taht the dummies are dummies
		foreach dummy in `time' `treat' {

			cap assert inlist(`dummy',1,0) | missing(`dummy') | `samplevar' == 0
			if _rc {
				noi di as error "{phang}In the difference in differnce regression for outputvar [`outputvar'], the dummy [`dummy'] is not a real dummy as it has values differnt from 1 and 0. See tab below. The tab is restricted to the observations that are not excluded due to missing values etc. in the difference in differnce regression.{p_end}""
				noi tab `dummy' if `samplevar' == 1, missing
				error 480
			}
		}

		**Test that for only two of three dummies there are observations
		* that has only that dummy. I.e. the two that is not the
		* interaction. If the interaction is 1, all three shluld be 1.

		*Test that there is some observations in each group
		forvalues tmt01 = 0/1 {
			forvalues t01 = 0/1 {

				*Summary stats on this group
				qui count if `treat' == `t01' & `time' == `tmt01' & `samplevar' == 1

				//There got to be more than 1 observation in each group for the regression to make sense
				if `r(N)' < 2 {
					noi di as error "{phang}In the difference in differnce regression for outputvar [`outputvar'] there were not enough observations in each combination of treatment [`treat'] and time [`time']. The tab below is restricted to the observations that are not excluded due to missing values etc. in the difference in differnce regression.{p_end}""
					noi tab `treat' `time' if `samplevar' == 1, missing
					error 480

				}
			}
		}

	end

/***************************************
****************************************

	Write sub-commands for stats

****************************************
***************************************/

*Sets up template for the result matrix
cap program drop 	templateResultMatrix
	program define	templateResultMatrix, rclass

	local 2ndDiff_cols 			2D 2D_err 2D_stars 2D_N

	local 1stDiff_C_cols		1DC 1DC_err 1DC_stars 1DC_N
	local 1stDiff_T_cols		1DT 1DT_err 1DT_stars 1DT_N

	local basicMean_C0_cols		C0_mean C0_err C0_N
	local basicMean_T0_cols		T0_mean T0_err T0_N
	local basicMean_C1_cols		C1_mean C1_err C1_N
	local basicMean_T1_cols		T1_mean T1_err T1_N

	local colnames `2ndDiff_cols' `1stDiff_C_cols' `1stDiff_T_cols' `basicMean_C0_cols' `basicMean_T0_cols' `basicMean_C1_cols' `basicMean_T0_cols'

	*Define default row here. The results for each var will be one row that starts with all missing vlaues
	mat startRow = (.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.)
	mat colnames startRow = `colnames'

	return matrix startRow startRow
	return local colnames `colnames'

end

*Take the significance levels and count number of starts
cap program drop 	countStars
	program define	countStars, rclass

	args pvalue star1 star2 star3

	*Test if custom star levels are used
	if "`star1'" == "" {

		*Set default levels for 1, 2 and 3 stars
		local star1 .1
		local star2 .05
		local star3 .01
	}

	local stars 0
	foreach star_p_level in `star1' `star2' `star3' {

		if `pvalue' < `star_p_level' local ++stars
	}

	return local stars `stars'

end

*Take the significance levels and count number of starts
cap program drop 	convertErrs
	program define	convertErrs, rclass

	args se_err N errortype

	*Test if custom star levels are used
	if "`errortype'" == "se" {
		local err `se_err'
	}
	else if "`errortype'" == "sd" {
		local err = `se_err' * sqrt(`N')
	}

	return local converted_error `err'

end

}

/***************************************
****************************************

	Write sub-commands for outputs

****************************************
***************************************/

	******
	*Then the result matrix can be passed into subcommands that output in either Excel, LaTeX or in the main window.

	*The results can be accessed like this, which makes the sub-commands less sensitive to changing column order in the section above.
	//mat A = ddtab_resultMap[3, "C0_Mean"] // returns a 1x1 matrix with the baseline mean for the countrol grop for the thrid outcome var
	//local a = el(A,1,1)				// returns the value

	//matlist A
	//di `a'

	cap program drop 	outputwindow
		program define	outputwindow

		/*
		Todo:
			note on stars
			Add errors on second line

		*/

		syntax varlist , ddtab_resultMap(name) labmaxlen(numlist) rwlbls(string) [covariates(string)]

		local numVars = `:word count `varlist''

		local statlist 2D 1DT 1DC C0_mean T0_mean
		local diformat = "%9.2f"

		local first_hhline = 2 + `labmaxlen'
		local first_col = 4 + `labmaxlen'

		local bsln_space = 2
		local diff_space = 3

		local bsln_width = ((`bsln_space' * 2) + 8)
		local diff_width = ((`diff_space' * 2) + 10)

		local ctrl_hline = `bsln_width' + 1 + `diff_width'
		local ctrl_space = (`ctrl_hline' - 7) / 2

		local tmt_hline = `bsln_width' + 1 + `diff_width'
		local tmt_space = (`tmt_hline' - 9) / 2

		noi di as text "{c TLC}{hline `first_hhline'}{c TT}{hline `ctrl_hline'}{c TT}{hline `tmt_hline'}{c TT}{hline 17}{c TRC}"
		noi di as text "{c |}{col `first_col'}{c |}{dup `ctrl_space': }Control{dup `ctrl_space': }{c |}{dup `tmt_space': }Treatment{dup `tmt_space': }{c |}  Difference-in  {c |}"
		noi di as text "{c |}{col 3}Variable{col `first_col'}{c |}{dup `bsln_space': }Baseline{dup `bsln_space': }{c |}{dup `diff_space': }Difference{dup `diff_space': }{c |}{dup `bsln_space': }Baseline{dup `bsln_space': }{c |}{dup `diff_space': }Difference{dup `diff_space': }{c |}   -difference   {c |}"
		noi di as text "{c LT}{hline `first_hhline'}{c +}{hline `bsln_width'}{c +}{hline `diff_width'}{c +}{hline `bsln_width'}{c +}{hline `diff_width'}{c +}{hline 17}{c RT}"

		//noi di "diff w : `diff_width'"
		//noi di "bsln w : `bsln_width'"

		forvalues row = 1/`numVars' {

			local rwlbls = trim(subinstr("`rwlbls'", "@@","", 1))
			gettoken label rwlbls : rwlbls, parse("@@")

			foreach stat of local statlist {

				displayformatter , statname("`stat'") row(`row') ddtab_resultMap(ddtab_resultMap) diformat("`diformat'") bslnw(`bsln_width') diffw(`diff_width')
				local `stat' `r(disp_stata)'
				local `stat'_space = `r(disp_pre_space)'

				//noi di "`stat' : ``stat'' : `r(disp_len)' : `r(disp_pre_space)'"
			}

			*Disaplay each variable row at the same time
			noi di as text "{c |} `label'{col `first_col'}{c |}{dup `C0_mean_space': }`C0_mean'{dup `1DC_space': }`1DC'{dup `T0_mean_space': }`T0_mean'{dup `1DT_space': }`1DT'{dup `2D_space': }`2D'"

		}

		noi di as text "{c BLC}{hline `first_hhline'}{c BT}{hline `bsln_width'}{c BT}{hline `diff_width'}{c BT}{hline `bsln_width'}{c BT}{hline `diff_width'}{c BT}{hline 17}{c BRC}"

		if ("`covariates'" != "") {

			noi di as text "  The following variables was included as covariates [`covariates']"

		}

	end

cap program drop 	displayformatter
	program define	displayformatter, rclass

	syntax , statname(string) row(numlist) ddtab_resultMap(name) diformat(string)  bslnw(numlist) diffw(numlist)

		mat temp = ddtab_resultMap[`row', "`statname'"]
		local `statname' = el(temp,1,1)
		local `statname' 	: display `diformat' ``statname''

		if inlist("`statname'", "2D", "1DT", "1DC") {

			mat starNumMat = ddtab_resultMap[`row', "`statname'_stars"]
			local starNum = el(starNumMat,1,1)

			if `starNum' == 0 local `statname' "``statname''   "
			if `starNum' == 1 local `statname' "``statname''*  "
			if `starNum' == 2 local `statname' "``statname''** "
			if `starNum' == 3 local `statname' "``statname''***"
		}

		local `statname' = ltrim("``statname''")

		local len = strlen("``statname''")
		//noi di "|``statname''|"
		//noi di `len'

		local numSpace 0

		if inlist("`statname'", "C0_mean", "T0_mean") {
			local numSpace = `bslnw' - `len' - 1
		}
		else if inlist("`statname'", "1DT", "1DC") {
			local numSpace = `diffw' - `len' - 1
		}
		else if inlist("`statname'", "2D") {
			local numSpace = 16 - `len'
		}

		if `numSpace' < 0 local numSpace 0

		return local disp_pre_space = `numSpace'
		return local disp_len = `len'
		return local disp_stata "``statname'' {c |}"

	end