iematch : helpfile

iematch : helpfile

{smcl}
{* }{...}
{hline}
help for {hi:iematch}
{hline}

{title:Title}

{phang2}{cmdab:iematch} {hline 2} Matching base observations towards target observations using on a single continous variable.

{title:Syntax}

{phang2}
{cmdab:iematch}
, {cmdab:grp:dummy(}{it:varname}{cmd:)} {cmdab:match:var(}{it:varname}{cmd:)}
[{cmdab:id:var(}{it:varlist}{cmd:)} {cmdab:m1} {cmdab:maxdiff(}{it:numlist}{cmd:)} {cmd:seedok}
{cmdab:matchid:name(}{it:string}{cmd:)} {cmdab:matchdi:ffname(}{it:string}{cmd:)}
{cmdab:matchre:sultname(}{it:string}{cmd:)} {cmdab:matchco:untname(}{it:string}{cmd:)}
]

{marker opts}{...}
{synoptset 22}{...}
{synopthdr:options}
{synoptline}
{synopt :{cmdab:grp:dummy(}{it:varname}{cmd:)}}The group dummy variable where 1
	indicates base observations and 0 target observations{p_end}
{synopt :{cmdab:match:var(}{it:varname}{cmd:)}}The variable with a continous value
	to match on{p_end}
{synopt :{cmdab:id:var(}{it:varlist}{cmd:)}}The uniquely and fully identifying ID
	varaible. Used to indicate which target observation a base observation is match
	with. If omitted an ID variable will be created.{p_end}
{synopt :{cmdab:m1}}Allows many-to-one matches. The default is to allow only
	one-to-one matches. See the {help iematch##desc:description} section.{p_end}
{synopt :{cmdab:maxdiff(}{it:numlist}{cmd:)}}Set a maximum difference allowed in
	{cmdab:matchvar()}. If a base observation has no match within this difference
 then it will remain unmatched{p_end}
{synopt :{cmd:seedok}}Supresses the error maessage thrown when there are duplicates
	in {cmd:matchvar()}. When there are duplicates, the seed needs to be set in order
	to have a replicable match. The {help seed} should be set before this command.{p_end}
{synopt :{cmdab:matchid:name(}{it:string}{cmd:)}}Manually sets the name of the
	variable that indicates which target observation each base observation is
	matched with. The default is _matchID{p_end}
{synopt :{cmdab:matchdi:ffname(}{it:string}{cmd:)}}Manually sets the name of the
	variable that indicates the differnece in {cmdab:matchvar()} in each match
	pair/group. The default is _matchDiff{p_end}
{synopt :{cmdab:matchre:sultname(}{it:string}{cmd:)}}Manually sets the name of
	the variable that indicates if an observation was matched, or provide a reason why
	it was not. The default is _matchOrReason{p_end}
{synopt :{cmdab:matchco:untname(}{it:string}{cmd:)}}Manually sets the name of the
	variable that indicates how many observations a target obsersvation is matched
	with in a many-to-one matches. The default is _matchCount{p_end}
{synoptline}

{marker desc}
{title:Description}

{pstd}{cmdab:iematch} matches base observations towards target observations in terms
	of nearest value in {cmd:matchvar()}. Base observations are observations with
	value 1 in {cmd:grpdummy()} and target observations are observations with value
	0. For example, in a typical p-score match, base observations are treatment and
	target is control, however, there are many other examples of matching where it could
	be different.

{pstd}{cmdab:iematch} base its matching algorithm on the Stable Marriage algorithm.
	This algorithm was chosen because it always a solution and because it was possible
	to implement a stable matching even if multiple observations has identical match
	values (see {cmd:seed()} option for more details). One disadvantage is of this
	algorithm is that it only take local efficiency into account and not global. This
	means that it is possible that other matching algorithms might generate a more
	efficient match in terms of the sum of the difference of all matched pairs/groups.

{pstd}{hi:{ul:One-to-one matching algorithm}}{break}
	The algorithm used in a one-to-one match starts by evaluating which
	target observation each base observation is closest to and vice versa for
	target observations. If a base and a target observation pair mutually prefer
	each other, then those two observations are a match. The algorithm then repeats
	these two steps but excludes observations after they are matched until all base
	observations are either matched or excluded due to the option {cmdab:maxdiff()}.
	Matched observations ends up in pairs of exactly one base observation and one
	target observation.

{pstd}In a one-to-one match there have to be at least as many target observations
	as there are base observation. A one-to-one match returns three variables. One
	variable indicates the result of the matching for each observation. For example,
	matched, not matched, no match within the max difference etc. The other two variables
	hold info on the matched pair. One variable is the ID of the target observation
	in the match pair, and the other variable is the difference in {cmd:matchvar()}
	between the two observation in the match pair. See the below in this section
	for more details on the variables generated.

{pstd}{hi:{ul:Many-to-one matching algorithm}}{break}
	The algorithm used here is the same as in a one-to-one match. But instead
	of matching only when there is mutual preference, all base observations are
	matched towards their preferred target observation in the first step, as long
	as the match is within the max-value if {cmdab:maxdff()} is used. Matched observations
	end up in groups in which there is exactly one target observation but where there are
	either one or several base observations.

{pstd}In a many-to-one match there is no restriction in terms of the number of
	target observations in relation to base observations. The many-to-one matching
	results in four variables. Three of the variables are the same as in a one-to-one
	match. The fourth variable indicates how many base observations that were matched
	towards each target observation. See the below in this section for more details on
	the variables generated.

{pstd}{hi:{ul:Variables generated}}{break}
	This section explains the variables that are generated by this command. All variables
	will be referred to by their default names, but those names can be manually set to somthing
	else using the corresponding options.

{pstd}{hi:_matchID}{break}
	This variable indicates the ID if the target observations in each match pair/group. For matched
	target observations, the value in this variable will be the same value as in teh ID
	variable. Since the values in the ID varaible are unique and there is exactly one target
	observation in each matched pair/group, this variable function as a unique pair/group ID.
	In addition to indicating which observations are included in the same pair/group, this
	varaible can be used to include a pair/group fixed effect in a regression.

{pstd}{hi:_matchDiff}{break}
	This variable indicates the difference in {cmd:matchvar()} between matched base observations
	and target observations. In a one-to-one match this value is the identical for
	both the base observation and the target observation in each matched pair.
	In a many-to-one match, this value is only indicated for base observations. It is
	missing for target observations as there are potentially multiple matches,. and
	then there are not one single difference.

{pstd}{hi:_matchOrReason}{break}
	This variable indicates the result of the match for all observations. Observations
	that ended up in a match pair or group has the value 1. Target observations
	not matched has the value 0. Any base observation without a valid match due to {cmdab:maxdff()} are
	assigned the value .n with a descriptive label. Any observations excluded from the match completely,
	either by having a missing value in {cmd:grpdummy()} or by using {inp:if} or {inp:in} are
	assigned the value .m and a descriptive label.

{pstd}{hi:_matchCount}{break}
	In a many-to-one match, this variable indicates how many target observations were
	matched towards each matched base observation. This variable can be used as regression
	weights when controlling for the fact that some observations are matched towards multiple
	observations.

{marker optslong}
{title:Options}

{phang}{cmdab:grp:dummy(}{it:varname}{cmd:)} is the dummy variable that indicates if an observation
	is a base or target observation. This variable, must be numeric and is only allowed to have
	the values 1, 0 or missing. 1 indicates a base observation, 0 indicates a target observation,
	and observations with a missing value will be excluded from the missing.

{phang}{cmdab:match:var(}{it:varname}{cmd:)} is the convariable used in the
	matching. This must be a numeric variable, and it is typically a continuous
	variable. This variable must not be missing for any observations that has
	either the value 1 or 0 in the group dummy variable.

{phang}{cmdab:id:var(}{it:varlist}{cmd:)} indicates the variable that uniquely
	and fully identifies the data set. The values in this variable is the values
	that will be used in the variable that indicates which target observation each
	base observations matched against.

{phang}{cmdab:m1} set the match to a many-to-one match. This allows multiple base
	observations to be matched towards a single target observation. The default is
	a one-to-one match when each matched target observation only is mathced towards
	a single base observation. This option allows the number of base observations
	to be larger then the number of target observations. See {help iematch##desc:description}
	section for more details.

{phang}{cmdab:maxdiff(}{it:numlist}{cmd:)} sets a maximum allowed difference between
	a base observation and a target observation for a match to be valid. Any base
	observation without a match within this difference will end up unmatched.

{phang}{cmd:seedok} supresses the error maessage thrown when there are duplicates among
	the base observations or the target observations in {cmd:matchvar()}. When there
	are duplicates a random variable is used to distinguish which variable to match.
	Setting the seed makes this randomization replicable meaning that the matching
	is replicable. The {help seed} should be set before this command by the user. When the
	seed is set, or if replicable matching is not important, then the option {cmd:seedok} can
	be used to supress the error message. Duplicates between base observations and
	target observations are allowed.

{phang}{cmdab:matchid:name(}{it:string}{cmd:)} allows the user to manually set the
	name of the variable that indicates the target observations in each match pair/group.
	If omitted, the default name is {inp:_matchID}. The names {inp:_ID}, {inp:_matchDiff}, {inp:_matchOrReason}
	and {inp:_matchCount} are not allowed.

{phang}{cmdab:matchdi:ffname(}{it:string}{cmd:)} allows the user to manually set
	the name of the variable that indicates the difference between the matched base observations
	and target observations. If omitted, the default name is {inp:_matchDiff}. The
	names {inp:_ID}, {inp:_matchID}, {inp:_matchOrReason} and {inp:_matchCount} are
	not allowed.

{phang}{cmdab:matchre:sultname(}{it:string}{cmd:)} allows the user to manually set the
	name of the variable that indicates the result of the match for each observation.
	If omitted, the default name is {inp:_matchOrReason}. The names {inp:_ID},
	{inp:_matchID}, {inp:_matchDiff} and {inp:_matchCount} are not allowed.

{phang}{cmdab:matchco:untname(}{it:string}{cmd:)} allows the user to manually set the
	name of the variable that indicates the number of base observations that has
	been matched towards each matched matcehd target observation. This option may
	only be used in combination with option {cmd:m1}. If omitted, the default
	name is {inp:_matchCount}. The names {inp:_ID}, {inp:_matchID}, {inp:_matchDiff}
	and {inp:_matchOrReason} are not allowed.

{title:Author}

{phang}Kristoffer Bj�rkefur, The World Bank, DECIE

{phang}Please send bug-reports, suggestions and requests for clarifications
		 writing "ietools iematch" in the subject line to:{break}
		 kbjarkefur@worldbank.org

{phang}You can also see the code, make comments to the code, see the version
		 history of the code, and submit additions or edits to the code through
		 the github repository of ietoolkit:{break}
		 {browse "https://github.com/worldbank/ietoolkit"}

{smcl}
{* }{...}
{hline}
help for {hi:iematch}
{hline}

{title:Title}

{phang2}{cmdab:iematch} {hline 2} Matching base observations towards target observations using on a single continous variable.

{title:Syntax}

{phang2}
{cmdab:iematch}
, {cmdab:grp:dummy(}{it:varname}{cmd:)} {cmdab:match:var(}{it:varname}{cmd:)}
[{cmdab:id:var(}{it:varlist}{cmd:)} {cmdab:m1} {cmdab:maxdiff(}{it:numlist}{cmd:)} {cmd:seedok}
{cmdab:matchid:name(}{it:string}{cmd:)} {cmdab:matchdi:ffname(}{it:string}{cmd:)}
{cmdab:matchre:sultname(}{it:string}{cmd:)} {cmdab:matchco:untname(}{it:string}{cmd:)}
]

{marker opts}{...}
{synoptset 22}{...}
{synopthdr:options}
{synoptline}
{synopt :{cmdab:grp:dummy(}{it:varname}{cmd:)}}The group dummy variable where 1
	indicates base observations and 0 target observations{p_end}
{synopt :{cmdab:match:var(}{it:varname}{cmd:)}}The variable with a continous value
	to match on{p_end}
{synopt :{cmdab:id:var(}{it:varlist}{cmd:)}}The uniquely and fully identifying ID
	varaible. Used to indicate which target observation a base observation is match
	with. If omitted an ID variable will be created.{p_end}
{synopt :{cmdab:m1}}Allows many-to-one matches. The default is to allow only
	one-to-one matches. See the {help iematch##desc:description} section.{p_end}
{synopt :{cmdab:maxdiff(}{it:numlist}{cmd:)}}Set a maximum difference allowed in
	{cmdab:matchvar()}. If a base observation has no match within this difference
 then it will remain unmatched{p_end}
{synopt :{cmd:seedok}}Supresses the error maessage thrown when there are duplicates
	in {cmd:matchvar()}. When there are duplicates, the seed needs to be set in order
	to have a replicable match. The {help seed} should be set before this command.{p_end}
{synopt :{cmdab:matchid:name(}{it:string}{cmd:)}}Manually sets the name of the
	variable that indicates which target observation each base observation is
	matched with. The default is _matchID{p_end}
{synopt :{cmdab:matchdi:ffname(}{it:string}{cmd:)}}Manually sets the name of the
	variable that indicates the differnece in {cmdab:matchvar()} in each match
	pair/group. The default is _matchDiff{p_end}
{synopt :{cmdab:matchre:sultname(}{it:string}{cmd:)}}Manually sets the name of
	the variable that indicates if an observation was matched, or provide a reason why
	it was not. The default is _matchOrReason{p_end}
{synopt :{cmdab:matchco:untname(}{it:string}{cmd:)}}Manually sets the name of the
	variable that indicates how many observations a target obsersvation is matched
	with in a many-to-one matches. The default is _matchCount{p_end}
{synoptline}

{marker desc}
{title:Description}

{pstd}{cmdab:iematch} matches base observations towards target observations in terms
	of nearest value in {cmd:matchvar()}. Base observations are observations with
	value 1 in {cmd:grpdummy()} and target observations are observations with value
	0. For example, in a typical p-score match, base observations are treatment and
	target is control, however, there are many other examples of matching where it could
	be different.

{pstd}{cmdab:iematch} base its matching algorithm on the Stable Marriage algorithm.
	This algorithm was chosen because it always a solution and because it was possible
	to implement a stable matching even if multiple observations has identical match
	values (see {cmd:seed()} option for more details). One disadvantage is of this
	algorithm is that it only take local efficiency into account and not global. This
	means that it is possible that other matching algorithms might generate a more
	efficient match in terms of the sum of the difference of all matched pairs/groups.

{pstd}{hi:{ul:One-to-one matching algorithm}}{break}
	The algorithm used in a one-to-one match starts by evaluating which
	target observation each base observation is closest to and vice versa for
	target observations. If a base and a target observation pair mutually prefer
	each other, then those two observations are a match. The algorithm then repeats
	these two steps but excludes observations after they are matched until all base
	observations are either matched or excluded due to the option {cmdab:maxdiff()}.
	Matched observations ends up in pairs of exactly one base observation and one
	target observation.

{pstd}In a one-to-one match there have to be at least as many target observations
	as there are base observation. A one-to-one match returns three variables. One
	variable indicates the result of the matching for each observation. For example,
	matched, not matched, no match within the max difference etc. The other two variables
	hold info on the matched pair. One variable is the ID of the target observation
	in the match pair, and the other variable is the difference in {cmd:matchvar()}
	between the two observation in the match pair. See the below in this section
	for more details on the variables generated.

{pstd}{hi:{ul:Many-to-one matching algorithm}}{break}
	The algorithm used here is the same as in a one-to-one match. But instead
	of matching only when there is mutual preference, all base observations are
	matched towards their preferred target observation in the first step, as long
	as the match is within the max-value if {cmdab:maxdff()} is used. Matched observations
	end up in groups in which there is exactly one target observation but where there are
	either one or several base observations.

{pstd}In a many-to-one match there is no restriction in terms of the number of
	target observations in relation to base observations. The many-to-one matching
	results in four variables. Three of the variables are the same as in a one-to-one
	match. The fourth variable indicates how many base observations that were matched
	towards each target observation. See the below in this section for more details on
	the variables generated.

{pstd}{hi:{ul:Variables generated}}{break}
	This section explains the variables that are generated by this command. All variables
	will be referred to by their default names, but those names can be manually set to somthing
	else using the corresponding options.

{pstd}{hi:_matchID}{break}
	This variable indicates the ID if the target observations in each match pair/group. For matched
	target observations, the value in this variable will be the same value as in teh ID
	variable. Since the values in the ID varaible are unique and there is exactly one target
	observation in each matched pair/group, this variable function as a unique pair/group ID.
	In addition to indicating which observations are included in the same pair/group, this
	varaible can be used to include a pair/group fixed effect in a regression.

{pstd}{hi:_matchDiff}{break}
	This variable indicates the difference in {cmd:matchvar()} between matched base observations
	and target observations. In a one-to-one match this value is the identical for
	both the base observation and the target observation in each matched pair.
	In a many-to-one match, this value is only indicated for base observations. It is
	missing for target observations as there are potentially multiple matches,. and
	then there are not one single difference.

{pstd}{hi:_matchOrReason}{break}
	This variable indicates the result of the match for all observations. Observations
	that ended up in a match pair or group has the value 1. Target observations
	not matched has the value 0. Any base observation without a valid match due to {cmdab:maxdff()} are
	assigned the value .n with a descriptive label. Any observations excluded from the match completely,
	either by having a missing value in {cmd:grpdummy()} or by using {inp:if} or {inp:in} are
	assigned the value .m and a descriptive label.

{pstd}{hi:_matchCount}{break}
	In a many-to-one match, this variable indicates how many target observations were
	matched towards each matched base observation. This variable can be used as regression
	weights when controlling for the fact that some observations are matched towards multiple
	observations.

{marker optslong}
{title:Options}

{phang}{cmdab:grp:dummy(}{it:varname}{cmd:)} indicates which variable is the group
	dummy. This variable, must be numeric and is only allowed to have the values 1,
	0 or missing. 1 indicates a base observation, 0 indicates a target observation,
	and observations with a missing value will be excluded from the missing.

{phang}{cmdab:match:var(}{it:varname}{cmd:)} indicates the variable used in the
	matching. This must be a numeric variable, and it is typically a continuous
	variable. This variable must not be missing for any observations that has
	either the value 1 or 0 in the group dummy variable.

{phang}{cmdab:id:var(}{it:varlist}{cmd:)} indicates the variable that uniquely
	and fully identifies the data set. The values in this variable is the values
	that will be used in the variable that indicates which target observation each
	base observations matched against.

{phang}{cmdab:m1} set the match to a many-to-one match. This allows multiple base
	observations to be matched towards a single target observation. The default is
	a one-to-one match when each matched target observation only is mathced towards
	a single base observation. This option allows the number of base observations
	to be larger then the number of target observations. See {help iematch##desc:description}
	section for more details.

{phang}{cmdab:maxdiff(}{it:numlist}{cmd:)} sets a maximum allowed difference between
	a base observation and a target observation for a match to be valid. Any base
	observation without a match within this difference will end up unmatched.

{phang}{cmd:seedok} supresses the error maessage thrown when there are duplicates among
	the base observations or the target observations in {cmd:matchvar()}. When there
	are duplicates a random variable is used to distinguish which variable to match.
	Setting the seed makes this randomization replicable meaning that the matching
	is replicable. The {help seed} should be set before this command by the user. When the
	seed is set, or if replicable matching is not important, then the option {cmd:seedok} can
	be used to supress the error message. Duplicates between base observations and
	target observations are allowed.

{phang}{cmdab:matchid:name(}{it:string}{cmd:)} allows the user to manually set the
	name of the variable that indicates the target observations in each match pair/group.
	If omitted, the default name is {inp:_matchID}. The names {inp:_ID}, {inp:_matchDiff}, {inp:_matchOrReason}
	and {inp:_matchCount} are not allowed.

{phang}{cmdab:matchdi:ffname(}{it:string}{cmd:)} allows the user to manually set
	the name of the variable that indicates the difference between the matched base observations
	and target observations. If omitted, the default name is {inp:_matchDiff}. The
	names {inp:_ID}, {inp:_matchID}, {inp:_matchOrReason} and {inp:_matchCount} are
	not allowed.

{phang}{cmdab:matchre:sultname(}{it:string}{cmd:)} allows the user to manually set the
	name of the variable that indicates the result of the match for each observation.
	If omitted, the default name is {inp:_matchOrReason}. The names {inp:_ID},
	{inp:_matchID}, {inp:_matchDiff} and {inp:_matchCount} are not allowed.

{phang}{cmdab:matchco:untname(}{it:string}{cmd:)} allows the user to manually set the
	name of the variable that indicates the number of base observations that has
	been matched towards each matched matcehd target observation. This option may
	only be used in combination with option {cmd:m1}. If omitted, the default
	name is {inp:_matchCount}. The names {inp:_ID}, {inp:_matchID}, {inp:_matchDiff}
	and {inp:_matchOrReason} are not allowed.

{title:Author}

{phang}Kristoffer Bj�rkefur, The World Bank, DECIE

{phang}Please send bug-reports, suggestions and requests for clarifications
		 writing "ietools iematch" in the subject line to:{break}
		 kbjarkefur@worldbank.org

{phang}You can also see the code, make comments to the code, see the version
		 history of the code, and submit additions or edits to the code through
		 the github repository of ietoolkit:{break}
		 {browse "https://github.com/worldbank/ietoolkit"}


Update XX_FCS.do

Update XX_FCS.do

/*-------------------------------------------------------------------------------
# Name:		XX_FCS
# Purpose:	Create dietary diversity and food consumption scores for Uganda HH
# Author:	Tim Essam, Ph.D.
# Created:	10/31/2014; 02/19/2015.
# Owner:	USAID GeoCenter | OakStream Systems, LLC
# License:	MIT License
# Ado(s):	see below
#-------------------------------------------------------------------------------
*/
clear
capture log close
log using "$pathlog/fcs.log", replace
set more off

use "$pathraw/GSEC15B.dta", replace

* Fix observations who are missing ConsCategory information
* Export missing to spreadsheet, use concatenate command to write commands and order
tab itmcd if h15bq2c ==., sort

* Replace consumption category information with info based on type of food
replace h15bq2c = 1 if (inlist(itmcd, 110, 112, 113, 114, 115, 116, 138)==1 & h15bq2c ==.)
replace h15bq2c = 2 if (inlist(itmcd, 101, 102, 105, 107, 108, 109)==1 & h15bq2c ==.)
replace h15bq2c = 3 if itmcd == 147 & h15bq2c ==.
replace h15bq2c = 4 if (inlist(itmcd, 140, 141, 144)==1 & h15bq2c ==.)
replace h15bq2c = 6 if (inlist(itmcd, 135, 136, 137, 139)==1 & h15bq2c ==.)
replace h15bq2c = 7 if (inlist(itmcd, 130, 132, 133)==1 & h15bq2c ==.)
replace h15bq2c = 8 if (inlist(itmcd, 117, 118, 119, 122, 123, 124)==1 & h15bq2c ==.)
replace h15bq2c = 9 if itmcd == 125 & h15bq2c ==.
replace h15bq2c = 10 if (inlist(itmcd, 127, 129, 150)==1 & h15bq2c ==.)
replace h15bq2c = 11 if (inlist(itmcd, 148, 149, 151, 152, 153, 155, 156, 158)==1 & h15bq2c ==.)

* Create variables to calculate Food Consumption Score
egen cereal_days = max(h15bq3b) if inlist(h15bq2c, 1), by(HHID)
g cerealFCS = cereal_days * 2

egen starches_days = max(h15bq3b) if inlist(h15bq2c, 2), by(HHID)
g starchesFCS = starches_days * 2

* wheat, rice, cereal, starch
egen staples_days = max(h15bq3b) if inlist(h15bq2c, 1, 2), by(HHID)
g staplesFCS = staples_days * 2

* legumes, beans, lentils, nuts, peas & nuts and seeds
egen pulse_days = max(h15bq3b) if inlist(h15bq2c, 4, 5), by(HHID)
g pulseFCS = pulse_days * 3

* Both weighted by 1
egen veg_days = max(h15bq3b) if inlist(h15bq2c, 6), by(HHID)
g vegFCS = veg_days

egen fruit_days = max(h15bq3b) if inlist(h15bq2c, 7), by(HHID)
g fruitFCS = fruit_days

* meat, poultry, fish, eggs
egen meat_days = max(h15bq3b) if inlist(h15bq2c, 8), by(HHID)
g meatFCS = meat_days * 4

egen milk_days = max(h15bq3b) if inlist(h15bq2c, 9), by(HHID)
g milkFCS = milk_days * 4

egen sugar_days = max(h15bq3b) if inlist(h15bq2c, 3), by(HHID)
g sugarFCS = sugar_days * 0.5

egen oil_days = max(h15bq3b) if inlist(h15bq2c, 10), by(HHID)
g oilFCS = oil_days * 0.5

* Check "Outside food and drink" to ensure that you are not missing major consumables
tab itmcd if h15bq2c == 12

* Label the variables, get their averages and plot them on same graph to compare
local ftype cereal starches staples pulse veg fruit meat milk sugar oil
local n: word count `ftype'
forvalues i = 1/`n' {
	local a: word `i' of `ftype'
	la var `a'_days "Number of days consuming `a'"
	la var `a'FCS "Food consumption score for `x'"
	replace `a'_days = 0 if `a'_days == .
	replace `a'FCS = 0 if `a'FCS == .
}
*end

/* Create dietary diversity variable consisting of following food groups:
  1) Cereals - cereal_days & pulses (x)
  2) White roots and tubers - Starches (x)
  3) Vegetables (x)
  4) Fruits (x)
  5) Meat (x)
  6) Eggs (x)
  7) Fish and other seafood (x)
  8) Legumes, nuts and seeds (x)
  9) Milk and milk products (x)
  10) Oils an fats
  11) Sweets*
  12) Apices condiments and beverages	*/

local dietDiv cereal starches fruit veg pulse sugar milk
foreach x of local dietDiv {
	g byte `x' = inlist(`x'_days, 0, .) != 1
}
*end

g byte eggs	= (inlist(h15bq3b, 1, 2, 3, 4, 5, 6, 7) & h15bq2c == 8 & itmcd == 124)
g byte meat	= (inlist(h15bq3b, 1, 2, 3, 4, 5, 6, 7) & h15bq2c == 8 & inlist(itmcd, 122, 123, 124)!=1)
g byte fish	= (inlist(h15bq3b, 1, 2, 3, 4, 5, 6, 7) & h15bq2c == 8 & inlist(itmcd, 122, 123)==1)
g byte oil 	= (inlist(h15bq3b, 1, 2, 3, 4, 5, 6, 7) & h15bq2c == 10 & inlist(itmcd, 150) != 1)
g byte cond = (inlist(h15bq3b, 1, 2, 3, 4, 5, 6, 7) & h15bq2c == 10 & inlist(itmcd, 150) == 1)

local dietLab cereal starches fruit veg pulse sugar milk eggs meat fish oil cond
foreach x of local dietLab {
	la var `x' "Consumed `x' in last 7 days"
}
*end

* Check  househlolds not reporting any consumption
egen tmp = rsum(cereal starches fruit veg pulse sugar milk eggs meat fish oil cond)
egen tmpsum = total(tmp), by(HHID)
* for checking which HH are missing all consumption information
* br if tmpsum == 0
drop tmp tmpsum

* Keep derived data (FCS & dietary diversity scores) and HHID
ds(h15* itmcd untcd), not
keep `r(varlist)'

* Collapse down to household level using max option, retain labels
qui include "$pathdo/copylabels.do"
ds(HHID), not
collapse (max) `r(varlist)', by(HHID)
qui include "$pathdo/attachlabels.do"

* Create food consumption score (FCS) and dietary diversity variables
egen FCS = rsum2(staplesFCS pulseFCS vegFCS fruitFCS meatFCS milkFCS sugarFCS oilFCS)
egen dietDiv = rsum(cereal starches fruit veg pulse sugar milk eggs meat fish oil cond)

* Clone FCS and recode based on Bangladesh thresholds
clonevar FCS_categ = FCS
recode FCS_categ (0/21 = 0) (21.5/35 = 1) (35.1/53 = 2) (53/112 = 3)
lab def fcscat 0 "Poor" 1 " Borderline" 2 " Acceptable low" 3 "Acceptable high"
lab val FCS_categ fcscat
la var FCS_categ "Food consumption score category"
tab FCS_cat, mi

* Some households have scores of zero which means they didn't eat or values are miscoded
sum *FCS dietDiv
assert FCS <= 112
assert dietDiv <= 12
la var FCS "Food Consumption Score"
la var dietDiv "Dietary diversity (12 food groups)"
g byte foodTag = inlist(dietDiv, 0, 1) == 1

save "$pathout/fstmp.dta", replace

* Load module on fortified food consumption
use "$pathraw/GSEC15BB", replace

g byte foodFtfd = (h15bq14 == 1 & h15bq15 == 1)
la var foodFtfd "HH consumes fortified food"

g byte maizeFtfd = (h15bq14 == 1 & h15bq15 == 1 & h15bqid == 113)
g byte oilFtfd 	 = (h15bq14 == 1 & h15bq15 == 1 & h15bqid == 127)
g byte sugarFtfd = (h15bq14 == 1 & h15bq15 == 1 & h15bqid == 147)
g byte saltFtfd  = (h15bq14 == 1 & h15bq15 == 1 & h15bqid == 150)

la var maizeFtfd "HH consumes fortified maize"
la var oilFtfd 	 "HH consumes fortified oil"
la var sugarFtfd "HH consumes fortified oil"
la var saltFtfd  "HH consumes fortified salt"

* keep new variables
ds(h15* result_code), not
keep `r(varlist)'

qui include "$pathdo/copylabels.do"
ds(HHID), not
collapse (max) `r(varlist)', by(HHID)
qui include "$pathdo/attachlabels.do"

* Merge with other food consumption data
merge 1:1 HHID using "$pathout/fstmp.dta", gen(food_merge)
drop if food_merge ==1
drop food_merge
compress

save "$pathout/fstmp2.dta", replace

* Bring in food security information (pp 32)
use "$pathraw/GSEC17A.dta", clear

* Create baseic welfare indicators
g byte clothing =  h17q2 == 1
la var clothing "Every hh member owns 2 sets of clothing"

g byte blanket = h17q3 == 1
la var blanket "Every child in hh has a blanket"

g byte shoes = h17q4 == 1
la var shoes "Every member has at least 1 pair of shoes"

clonevar mealsTaken = h17q5
recode mealsTaken (10 = .) (14 = .)

clonevar saltShortage = h17q6
clonevar bkfst_ychild = h17q7
clonevar bkfst_child = h17q7

clonevar foodLack = h17q9

* Keep new variables and HHID
ds(h17* result_code), not
keep `r(varlist)'

* Merge with other diet information data
merge 1:1 HHID using "$pathout/fstmp2.dta"
compress
save "$pathout/foodSecurity.dta", replace
erase "$pathout/fstmp.dta"
erase "$pathout/fstmp2.dta"

* Merge in Geovars for R graphics
merge 1:1 HHID using "$pathout/Geovars.dta", gen(geo_merge)
drop geo_merge

* Check correlation of FCS and Dietary Diversity
twoway (lpolyci FCS dietDiv if foodTag!=1, /*
*/fitplot(connected)), ytitle(Food Consumption Score) /*
*/xtitle(Dietary Diversity) title(FCS & Dietary Diversity/*
*/ Correlation) legend(order(1 "95% CI" 2 "Local Polynomial Smoothed")) scheme(mrc)
pwcorr FCS dietDiv, sig

* Export a cut of FCS and dietary diversity to R for plotting
preserve
keep if foodTag != 1
keep staples_days pulse_days milk_days meat_days veg_days oil_days sugar_days /*
*/ fruit_days FCS HHID region urban subRegion hid
rename *_days* *
order oil staples veg pulse sugar milk meat fruit FCS
export delimited using "$pathexport/food.consumption.score.csv", replace
restore

preserve
keep if foodTag != 1
keep dietDiv hid region subRegion
export delimited using "$pathexport/diet.diversity.csv", replace
restore

/*-------------------------------------------------------------------------------
# Name:		XX_FCS
# Purpose:	Create dietary diversity and food consumption scores for Uganda HH
# Author:	Tim Essam, Ph.D.
# Created:	10/31/2014; 02/19/2015.
# Owner:	USAID GeoCenter | OakStream Systems, LLC
# License:	MIT License
# Ado(s):	see below
#-------------------------------------------------------------------------------
*/
clear
capture log close
log using "$pathlog/fcs.log", replace
set more off

use "$pathraw/GSEC15B.dta", replace

* Fix observations who are missing ConsCategory information
* Export missing to spreadsheet, use concatenate command to write commands and order
tab itmcd if h15bq2c ==., sort

* Replace consumption category information with info based on type of food
replace h15bq2c = 1 if (inlist(itmcd, 110, 112, 113, 114, 115, 116, 138)==1 & h15bq2c ==.)
replace h15bq2c = 2 if (inlist(itmcd, 101, 102, 105, 107, 108, 109)==1 & h15bq2c ==.)
replace h15bq2c = 3 if itmcd == 147 & h15bq2c ==.
replace h15bq2c = 4 if (inlist(itmcd, 140, 141, 144)==1 & h15bq2c ==.)
replace h15bq2c = 6 if (inlist(itmcd, 135, 136, 137, 139)==1 & h15bq2c ==.)
replace h15bq2c = 7 if (inlist(itmcd, 130, 132, 133)==1 & h15bq2c ==.)
replace h15bq2c = 8 if (inlist(itmcd, 117, 118, 119, 122, 123, 124)==1 & h15bq2c ==.)
replace h15bq2c = 9 if itmcd == 125 & h15bq2c ==.
replace h15bq2c = 10 if (inlist(itmcd, 127, 129, 150)==1 & h15bq2c ==.)
replace h15bq2c = 11 if (inlist(itmcd, 148, 149, 151, 152, 153, 155, 156, 158)==1 & h15bq2c ==.)

* Create variables to calculate Food Consumption Score
egen cereal_days = max(h15bq3b) if inlist(h15bq2c, 1), by(HHID)
g cerealFCS = cereal_days * 2

egen starches_days = max(h15bq3b) if inlist(h15bq2c, 2), by(HHID)
g starchesFCS = starches_days * 2

* wheat, rice, cereal, starch
egen staples_days = max(h15bq3b) if inlist(h15bq2c, 1, 2), by(HHID)
g staplesFCS = staples_days * 2

* legumes, beans, lentils, nuts, peas & nuts and seeds
egen pulse_days = max(h15bq3b) if inlist(h15bq2c, 4, 5), by(HHID)
g pulseFCS = pulse_days * 3

* Both weighted by 1
egen veg_days = max(h15bq3b) if inlist(h15bq2c, 6), by(HHID)
g vegFCS = veg_days

egen fruit_days = max(h15bq3b) if inlist(h15bq2c, 7), by(HHID)
g fruitFCS = fruit_days

* meat, poultry, fish, eggs
egen meat_days = max(h15bq3b) if inlist(h15bq2c, 8), by(HHID)
g meatFCS = meat_days * 4

egen milk_days = max(h15bq3b) if inlist(h15bq2c, 9), by(HHID)
g milkFCS = milk_days * 4

egen sugar_days = max(h15bq3b) if inlist(h15bq2c, 3), by(HHID)
g sugarFCS = sugar_days * 0.5

egen oil_days = max(h15bq3b) if inlist(h15bq2c, 10), by(HHID)
g oilFCS = oil_days * 0.5

* Check "Outside food and drink" to ensure that you are not missing major consumables
tab itmcd if h15bq2c == 12

* Label the variables, get their averages and plot them on same graph to compare
local ftype cereal starches staples pulse veg fruit meat milk sugar oil
local n: word count `ftype'
forvalues i = 1/`n' {
	local a: word `i' of `ftype'
	la var `a'_days "Number of days consuming `a'"
	la var `a'FCS "Food consumption score for `x'"
	replace `a'_days = 0 if `a'_days == .
	replace `a'FCS = 0 if `a'FCS == .
}
*end

/* Create dietary diversity variable consisting of following food groups:
  1) Cereals - cereal_days & pulses (x)
  2) White roots and tubers - Starches (x)
  3) Vegetables (x)
  4) Fruits (x)
  5) Meat (x)
  6) Eggs (x)
  7) Fish and other seafood (x)
  8) Legumes, nuts and seeds (x)
  9) Milk and milk products (x)
  10) Oils an fats
  11) Sweets*
  12) Apices condiments and beverages	*/

local dietDiv cereal starches fruit veg pulse sugar milk
foreach x of local dietDiv {
	g byte `x' = inlist(`x'_days, 0, .) != 1
}
*end

g byte eggs	= (inlist(h15bq3b, 1, 2, 3, 4, 5, 6, 7) & h15bq2c == 8 & itmcd == 124)
g byte meat	= (inlist(h15bq3b, 1, 2, 3, 4, 5, 6, 7) & h15bq2c == 8 & inlist(itmcd, 122, 123, 124)!=1)
g byte fish	= (inlist(h15bq3b, 1, 2, 3, 4, 5, 6, 7) & h15bq2c == 8 & inlist(itmcd, 122, 123)==1)
g byte oil 	= (inlist(h15bq3b, 1, 2, 3, 4, 5, 6, 7) & h15bq2c == 10 & inlist(itmcd, 150) != 1)
g byte cond = (inlist(h15bq3b, 1, 2, 3, 4, 5, 6, 7) & h15bq2c == 10 & inlist(itmcd, 150) == 1)

local dietLab cereal starches fruit veg pulse sugar milk eggs meat fish oil cond
foreach x of local dietLab {
	la var `x' "Consumed `x' in last 7 days"
}
*end

* Check  househlolds not reporting any consumption
egen tmp = rsum(cereal starches fruit veg pulse sugar milk eggs meat fish oil cond)
egen tmpsum = total(tmp), by(HHID)
* for checking which HH are missing all consumption information
* br if tmpsum == 0
drop tmp tmpsum

* Keep derived data (FCS & dietary diversity scores) and HHID
ds(h15* itmcd untcd), not
keep `r(varlist)'

* Collapse down to household level using max option, retain labels
qui include "$pathdo2/copylabels.do"
ds(HHID), not
collapse (max) `r(varlist)', by(HHID)
qui include "$pathdo2/attachlabels.do"

* Create food consumption score (FCS) and dietary diversity variables
egen FCS = rsum2(staplesFCS pulseFCS vegFCS fruitFCS meatFCS milkFCS sugarFCS oilFCS)
egen dietDiv = rsum(cereal starches fruit veg pulse sugar milk eggs meat fish oil cond)

* Clone FCS and recode based on Bangladesh thresholds
clonevar FCS_categ = FCS
recode FCS_categ (0/21 = 0) (21.5/35 = 1) (35.1/53 = 2) (53/112 = 3)
lab def fcscat 0 "Poor" 1 " Borderline" 2 " Acceptable low" 3 "Acceptable high"
lab val FCS_categ fcscat
la var FCS_categ "Food consumption score category"
tab FCS_cat, mi

* Some households have scores of zero which means they didn't eat or values are miscoded
sum *FCS dietDiv
assert FCS <= 112
assert dietDiv <= 12
la var FCS "Food Consumption Score"
la var dietDiv "Dietary diversity (12 food groups)"
g byte foodTag = inlist(dietDiv, 0, 1) == 1

save "$pathout/fstmp.dta", replace

* Load module on fortified food consumption
use "$pathraw/GSEC15BB", replace

g byte foodFtfd = (h15bq14 == 1 & h15bq15 == 1)
la var foodFtfd "HH consumes fortified food"

g byte maizeFtfd = 	(h15bq14 == 1 & h15bq15 == 1 & h15bqid == 113)
g byte oilFtfd 	 = 	(h15bq14 == 1 & h15bq15 == 1 & h15bqid == 127)
g byte sugarFtfd = 	(h15bq14 == 1 & h15bq15 == 1 & h15bqid == 147)
g byte saltFtfd  = 	(h15bq14 == 1 & h15bq15 == 1 & h15bqid == 150)

la var maizeFtfd "HH consumes fortified maize"
la var oilFtfd 	 "HH consumes fortified oil"
la var sugarFtfd "HH consumes fortified oil"
la var saltFtfd  "HH consumes fortified salt"

* keep new variables
ds(h15* result_code), not
keep `r(varlist)'

qui include "$pathdo2/copylabels.do"
ds(HHID), not
collapse (max) `r(varlist)', by(HHID)
qui include "$pathdo2/attachlabels.do"

* Merge with other food consumption data
merge 1:1 HHID using "$pathout/fstmp.dta", gen(food_merge)
drop if food_merge ==1
drop food_merge
compress

save "$pathout/fstmp2.dta", replace

* Bring in food security information (pp 32)
use "$pathraw/GSEC17A.dta", clear

* Create baseic welfare indicators
g byte clothing =  h17q2 == 1
la var clothing "Every hh member owns 2 sets of clothing"

g byte blanket = h17q3 == 1
la var blanket "Every child in hh has a blanket"

g byte shoes = h17q4 == 1
la var shoes "Every member has at least 1 pair of shoes"

clonevar mealsTaken = h17q5
recode mealsTaken (10 = .) (14 = .)

clonevar saltShortage = h17q6
clonevar bkfst_ychild = h17q7
clonevar bkfst_child = h17q7

clonevar foodLack = h17q9

* Keep new variables and HHID
ds(h17* result_code), not
keep `r(varlist)'

* Merge with other diet information data
merge 1:1 HHID using "$pathout/fstmp2.dta"
compress
save "$pathout/foodSecurity.dta", replace
erase "$pathout/fstmp.dta"
erase "$pathout/fstmp2.dta"

* Merge in Geovars for R graphics
merge 1:1 HHID using "$pathout/Geovars.dta", gen(geo_merge)
drop geo_merge

* Check correlation of FCS and Dietary Diversity
twoway (lpolyci FCS dietDiv if foodTag!=1, /*
*/fitplot(connected)), ytitle(Food Consumption Score) /*
*/xtitle(Dietary Diversity) title(FCS & Dietary Diversity/*
*/ Correlation) legend(order(1 "95% CI" 2 "Local Polynomial Smoothed")) scheme(mrc)
pwcorr FCS dietDiv, sig

* Export a cut of FCS and dietary diversity to R for plotting
preserve
keep if foodTag != 1
keep staples_days pulse_days milk_days meat_days veg_days oil_days sugar_days /*
*/ fruit_days FCS HHID region urban subRegion hid
rename *_days* *
order oil staples veg pulse sugar milk meat fruit FCS
export delimited using "$pathexport/food.consumption.score.csv", replace
restore

preserve
keep if foodTag != 1
keep dietDiv hid region subRegion
export delimited using "$pathexport/diet.diversity.csv", replace
restore


iedropone : help typos

iedropone : help typos

{smcl}
{* 26 Dec 2016}{...}
{hline}
help for {hi:iedropone}
{hline}

{title:Title}

{phang2}{cmdab:iedropone} {hline 2} Same function as {help drop} but prevents
	that additional observations are unintentionally dropped.
	See {help iedropone##examples:examples below}.

{title:Syntax}

{phang2}
{cmdab:iedropone} [if] , {cmdab:n:umobs(}{it:integer}{cmd:)}
	{cmdab:mvar(}{it:varname}{cmd:)} {cmdab:mval(}{it:list of values}{cmd:)}
	{cmd:zerook}]

{marker opts}{...}
{synoptset 22}{...}
{synopthdr:options}
{synoptline}
{synopt :{cmdab:n:umobs(}{it:integer}{cmd:)}}Number of observations that is allowed to be dropped. Default is 1.{p_end}
{synopt :{cmd:zerook}}Allows that no observation is dropped.{p_end}
{synopt :{cmdab:mvar(}{it:varname}{cmd:)}}Variable for which multiple values should be dropped. Must be used together with {cmdab:mval()}.{p_end}
{synopt :{cmdab:mval(}{it:list of values}{cmd:)}}The list of values in {cmdab:mvar()} that should be dropped. Must be used together with {cmdab:mvar()}.{p_end}
{synoptline}

{marker desc}
{title:Description}

{pstd}This commands might be easier to understand by following
	the {help iedropone##examples:examples below} before reading the
	description or the explanations of the options.

{pstd}{cmdab:iedropone} has the identical purpose as {help drop} when dropping
	observations. However, {cmdab:iedropone} safeguards that no additional
	observations are unintentionally dropped, or that changes are made to the
	data so that the observations that are supposed to be dropped are no longer dropped.

{pstd}{cmdab:iedropone} checks that no more or fewer observations than intended are
	dropped. For example, in the case that one observation has been identified to
	be dropped, then we want to make sure that when re-running the do-file
	no other observations are dropped even if more observations are added to that data
	set or changed in any other way.

{pstd}While the default is 1, {cmdab:iedropone} allows the user to set any another number
	of observation that should be dropped. If the number of observations that fit the
	drop condition is different, then the command will throw an error.

{marker optslong}
{title:Options}

{phang}{cmdab:n:umobs(}{it:integer}{cmd:)} this allows the user to set the
	number of observation that should be dropped. The default is 1 but any
	positive integer can be used. The command throws an error if any other number
	of observations match the drop condition.

{phang}{cmd:zerook} allows that no observations are dropped. The default is that
	an error is thrown if no observations are dropped.

{phang}{cmdab:mvar(}{it:varname}{cmd:)} and {cmdab:mval(}{it:list of values}{cmd:)} allows
	that multiple values in one variable are dropped. These two options must be used together.
	If the variable in {cmd:mvar()} is a string variable and some of the values in {cmd:mval()}
	includes spaces, then the list of values in {cmd:mval()} must be listed exactly as in example 4 below. The
	command loops over the values in {cmd:mval()} and drops the observations that
	satisfy the {it:if} condition and each of the value in {cmd:mval()}. For example:{p_end}

{pmore}{inp:iedropone if village == 100 , mvar(household_id) mval(21 22 23)}

{pmore}is identical to:

{pmore}{inp:iedropone if village == 100 & household_id == 21}{break}{inp:iedropone if village == 100 & household_id == 22}{break}{inp:iedropone if village == 100 & household_id == 23}

{pmore}The default is that exactly one observation should be dropped for each
	value in {cmd:mval()} unless {cmd:numobs()} or {cmd:zerook} is used. If those
	options are used then, then they apply to all values in {cmd:mval()} separately.

{phang}{cmdab:mval(}{it:list of values}{cmd:)}, see {cmdab:mvar(}{it:varname}{cmd:)} above.

{marker examples}
{title:Examples}

{pstd} {hi:Example 1.}

{pmore}{inp:iedropone if household_id == 712047}

{pmore}Let's say that we have identified the household with the ID 712047 to be
	incorrect and it should be dropped. Identical to {inp:drop if household_id == 712047}
	but it will test that exactly one observation is dropped each time the do-file runs.
	This guarantees that we will get an error message that no observation is dropped
	if someone makes a change to the ID. Otherwise we would unknowingly keep this
	incorrect observation in our data set.

{pmore}Similarly, if a new observation is added that is the correct household with ID 712047,
	then both observation would be dropped without warning if we would have
	used {inp:drop if household_id == 712047}. {cmd:iedropone} will make sure that
	our drop condition are applied as intended even if the data set is changed.

{pstd} {hi:Example 2.}

{pmore}{inp:iedropone if household_id == 712047 & household_head == "Bob Smith"}

{pmore}Let's say we have added a new household  with the ID 712047. In order to
	drop only one of those observations we must expand the if condition to
	indicate which one of them we want to drop.

{pstd} {hi:Example 3.}

{pmore}{inp:iedropone if household_id == 712047, numobs(2)}

{pmore}Let's say we added a new household  with the ID 712047 but we want to drop
	exactly both of them, then we can use the option {cmd:numobs()} like above.
	The command will now throw an error if not exactly two observations have the
	household ID 712047.

{pstd} {hi:Example 4.}

{pmore}{inp:iedropone if village == 100, mvar(household_head) mvar(`" "Bob Smith" "Ann Davitt" "Blessing Johnson" "')}

{pmore}If the values in {cmd:mvar()} are strings with empty spaces then then each
	value have to be enclosed in double qoutes and the full list needs to start
	with {inp:`"} and end with {inp:"'}.

{title:Acknowledgements}

{phang}I would like to acknowledge the help in testing and proofreading I
	received in relation to this command and help file from (in alphabetic
	order):{p_end}
{pmore}Paula Gonzalez-Martinez{break}Seungmin Lee{break}Mrijan Rimal{break}

{title:Author}

{phang}Kristoffer Bjarkefur, The World Bank, DECIE

{phang}Please send bug-reports, suggestions and requests for clarifications
		 writing "ietoolkit iedropone" in the subject line to:{break}
		 kbjarkefur@worldbank.org

{phang}You can also see the code, make comments to the code, see the version
		 history of the code, and submit additions or edits to the code through
		 the github repository of ietoolkit:{break}
		 {browse "https://github.com/worldbank/ietoolkit"}

{smcl}
{* 26 Dec 2016}{...}
{hline}
help for {hi:iedropone}
{hline}

{title:Title}

{phang2}{cmdab:iedropone} {hline 2} Same function as {help drop} but prevents that additional observations are unintentionally dropped. See {help iedropone##examples:examples below}.

{title:Syntax}

{phang2}
{cmdab:iedropone} [if] , {cmdab:n:umobs(}{it:integer}{cmd:)}
	{cmdab:mvar(}{it:varname}{cmd:)} {cmdab:mval(}{it:list of values}{cmd:)}
	{cmd:zerook}]

{marker opts}{...}
{synoptset 22}{...}
{synopthdr:options}
{synoptline}
{synopt :{cmdab:n:umobs(}{it:integer}{cmd:)}}Number of observations that is allowed to be dropped. Default is 1.{p_end}
{synopt :{cmd:zerook}}Allows that no observation is dropped.{p_end}
{synopt :{cmdab:mvar(}{it:varname}{cmd:)}}Variable for which multiple values should be dropped. Must be used together with {cmdab:mval()}.{p_end}
{synopt :{cmdab:mval(}{it:list of values}{cmd:)}}The list of values in {cmdab:mvar()} that should be dropped. Must be used together with {cmdab:mvar()}.{p_end}
{synoptline}

{marker desc}
{title:Description}

{pstd}This commands might be easier to understand by following
	the {help iedropone##examples:examples below} before reading the
	description or the eplinations of the options.

{pstd}{cmdab:iedropone} has the identical purpose as {help drop} when dropping
	observations. However, {cmdab:iedropone} safeguards that no additional
	observations are unintentionally dropped, or that changes are made to the
	data so that the observations that are supposed to be dropped are no longer dropped.

{pstd}{cmdab:iedropone} checks that no more or fewer observations than intended are
	dropped. For example, in the case that one observation has been identified to
	be dropped, then we want to make sure that when re-running the do-file
	no other observations are dropped even if more observations are added to that data
	set or changed in any other way.

{pstd}While the default is 1, {cmdab:iedropone} allows the user to set any another number
	of observation that should be dropped. If the number of observations that fit the
	drop condition is different, then the command will throw an error.

{marker optslong}
{title:Options}

{phang}{cmdab:n:umobs(}{it:integer}{cmd:)} this allows the user to set the
	number of observation that should be dropped. The default is 1 but any
	positive integer can be used. The command throws an error if any other number
	of observations match the drop condition.

{phang}{cmd:zerook} allows that no observations are dropped. The default is that
	an error is thrown if no observations are dropped.

{phang}{cmdab:mvar(}{it:varname}{cmd:)} and {cmdab:mval(}{it:list of values}{cmd:)} allows
	that multiple values in one variable are dropped. These two options must be used together.
	If the variable in {cmd:mvar()} is a string variable and some of the values in {cmd:mval()}
	includes spaces, then the list of values in {cmd:mval()} must be listed exactly as in example 4 below. The
	command loops over the values in {cmd:mval()} and drops the observations that
	satisfy the {it:if} condition and each of the value in {cmd:mval()}. For example:{p_end}

{pmore}{inp:iedropone if village == 100 , mvar(household_id) mval(21 22 23)}

{pmore}is identical to:

{pmore}{inp:iedropone if village == 100 & household_id == 21}{break}{inp:iedropone if village == 100 & household_id == 22}{break}{inp:iedropone if village == 100 & household_id == 23}

{pmore}The default is that exactly one observation should be dropped for each
	value in {cmd:mval()} unless {cmd:numobs()} or {cmd:zerook} is used. If those
	options are used then, then they apply to all values in {cmd:mval()} separately.

{phang}{cmdab:mval(}{it:list of values}{cmd:)}, see {cmdab:mvar(}{it:varname}{cmd:)} above.

{marker examples}
{title:Examples}

{pstd} {hi:Example 1.}

{pmore}{inp:iedropone if household_id == 712047}

{pmore}Let's say that we have identified the household with the ID 712047 to be
	incorrect and it should be dropped. Identical to {inp:drop if household_id == 712047}
	but it will test that exactly one observation is dropped each time the do-file runs.
	This guarantees that we will get an error message that no observation is dropped
	if someone makes a change to the ID. Otherwise we would unknowingly keep this
	incorrect observation in our data set.

{pmore}Similarly, if a new observation is added that is the correct household with ID 712047,
	then both observation would be dropped without warning if we would have
	used {inp:drop if household_id == 712047}. {cmd:iedropone} will make sure that
	our drop condition are applied as intended even if the data set is changed.

{pstd} {hi:Example 2.}

{pmore}{inp:iedropone if household_id == 712047 & household_head == "Bob Smith"}

{pmore}Let's say we have added a new household  with the ID 712047. In order to
	drop only one of those observations we must expand the if condition to
	indicate which one of them we want to drop.

{pstd} {hi:Example 3.}

{pmore}{inp:iedropone if household_id == 712047, numobs(2)}

{pmore}Let's say we added a new household  with the ID 712047 but we want to drop
	exactly both of them, then we can use the option {cmd:numobs()} like above.
	The command will now throw an error if not exactly two observations have the
	household ID 712047.

{pstd} {hi:Example 4.}

{pmore}{inp:iedropone if village == 100, mvar(household_head) mvar(`" "Bob Smith" "Ann Davitt" "Blessing Johnson" "')}

{pmore}If the values in {cmd:mvar()} are strings with empty spaces then then each
	value have to be enclosed in double qoutes and the full list needs to start
	with {inp:`"} and end with {inp:"'}.

{title:Acknowledgements}

{phang}I would like to acknowledge the help in testing and proofreading I
	received in relation to this command and help file from (in alphabetic
	order):{p_end}
{pmore}Paula Gonzalez-Martinez{break}Seungmin Lee{break}Mrijan Rimal{break}

{title:Author}

{phang}Kristoffer Bjarkefur, The World Bank, DECIE

{phang}Please send bug-reports, suggestions and requests for clarifications
		 writing "ietoolkit iedropone" in the subject line to:{break}
		 kbjarkefur@worldbank.org

{phang}You can also see the code, make comments to the code, see the version
		 history of the code, and submit additions or edits to the code through
		 the github repository of ietoolkit:{break}
		 {browse "https://github.com/worldbank/ietoolkit"}


Bump ftools dependency to fix #206

Bump ftools dependency to fix #206

*! version 5.8.1 03jun2020

program reghdfe, eclass
	* Intercept old+version and cache(clear) operation
	cap syntax, old [version CACHE(passthru)]
	if !c(rc) {
        	reghdfe_old, `version' `cache'
		exit
	}

	* Intercept old
	cap syntax anything(everything) [fw aw pw/], [*] old
	if !c(rc) {
		di as error "(running historical version of reghdfe)"
		if ("`weight'"!="") local weightexp [`weight'=`exp']
		reghdfe_old `anything' `weightexp', `options'
		exit
	}

	* Aux. subcommands
	cap syntax, [*]
	if inlist("`options'", "check", "compile", "reload", "update", "version", "requirements", "store_alphas") {
		if ("`options'"=="compile") loc args force
		if ("`options'"=="check") loc options compile
		if ("`options'"=="update") {
			loc args 1
			loc options reload
		}
		loc subcmd = proper("`options'")
		`subcmd' `args'
	}
	else if replay() {
		Replay `0'
	}
	else {
		Cleanup 0
		ms_get_version ftools, min_version("2.38.0")
		cap noi Estimate `0'
		Cleanup `c(rc)'
	}
end

program Compile
	args force

	* Check dependencies
	ftools, check // in case lftools.mlib does not exist or is outdated
	ms_get_version ftools, min_version("2.34.0")
	ms_get_version reghdfe // save local package_version
	loc list_objects "FixedEffects() fixed_effects() BipartiteGraph()"
	loc list_functions "reghdfe_*() transform_*() accelerate_*() panelmean() panelsolve_*() lsmr()"
	loc list_misc "weighted_quadcolsum() safe_divide() check_convergence() precompute_inv_xx() _st_data_wrapper()"
	// TODO: prefix everything with reghdfe_*

	ms_compile_mata, ///
		package(reghdfe) ///
		version(`package_version') ///
		fun("`list_objects' `list_functions' `list_misc'") ///
		verbose ///
		`force'
end

program Reload
	* Internal debugging tool.
	* Updates dependencies and reghdfe from local path or from github
	* Usage:
	* 	reghdfe, update // from c:\git\..
	* 	reghdfe, reload // from github

	args online
	if ("`online'" == "") loc online 0

	di as text _n "{bf:reghdfe: updating required packages}"
	di as text "{hline 64}"

	* -ftools- https://github.com/sergiocorreia/ftools/
	cap ado uninstall ftools
	if (`online') net install ftools, from("https://github.com/sergiocorreia/ftools/raw/master/src/")
	if (!`online') net install ftools, from("c:\git\ftools\src")
	di as text "{hline 64}"
	ftools, compile
	di as text "{hline 64}"

	* Update -reghdfe-
	di as text _n  _n "{bf:reghdfe: updating self}"
	di as text "{hline 64}"
	qui ado uninstall reghdfe
	if (`online') net install reghdfe, from("https://github.com/sergiocorreia/reghdfe/raw/master/src/")
	if (!`online') net install reghdfe, from("c:\git\reghdfe\src")
	qui which reghdfe
	di as text "{hline 64}"
	reghdfe, compile
	di as text "{hline 64}"

	* Cleaning up
	di as text _n "{bf:Note:} You need to run {stata program drop _all} now."
end

program Version
	which reghdfe
	Requirements
end

program Requirements
	di as text _n "Required packages installed?"
	loc reqs ftools
	// ivreg2 avar tuples group3hdfe
	if (c(stata_version)<13) loc reqs `reqs' boottest

	loc ftools_github "https://github.com/sergiocorreia/ftools/raw/master/src/"

	loc error 0

	foreach req of local reqs {
		loc fn `req'.ado
		cap findfile `fn'
		if (_rc) {
			loc error 1
			di as text "{lalign 20:- `req'}" as error "not" _c
			di as text "    {stata ssc install `req':install from SSC}" _c
			if inlist("`req'", "ftools") {
				loc github ``req'_github'
				di as text `"    {stata `"net install `req', from(`"`github'"')"':install from github}"'
			}
			else {
				di as text // newline
			}
		}
		else {
			di as text "{lalign 20:- `req'}" as text "yes"
		}
	}

	if (`error') exit 601
end

program Store_Alphas, eclass
	mata: st_local("save_any_fe", strofreal(HDFE.save_any_fe))
	assert inlist(`save_any_fe', 0, 1)
	if (`save_any_fe') {
		_assert e(depvar) != "", msg("e(depvar) is empty")
		_assert e(resid) != "", msg("e(resid) is empty")
		// we can't use -confirm var- because it might have TS operators
		fvrevar `e(depvar)', list
		confirm numeric var `e(resid)', exact
		tempvar d
		if (e(rank)) {
			qui _predict double `d' if e(sample), xb
		}
		else if (e(report_constant)) {
			gen double `d' = _b[_cons] if e(sample)
		}
		else {
			gen double `d' = 0 if e(sample)
		}
		qui replace `d' = `e(depvar)' - `d' - `e(resid)' if e(sample)

		mata: HDFE.store_alphas("`d'")
		drop `d'

		// Drop resid if we don't want to save it; and update e(resid)
		cap drop __temp_reghdfe_resid__
		if (!c(rc)) ereturn local resid
	}
end

program Cleanup
	args rc
	cap mata: mata drop HDFE
	cap mata: mata drop hdfe_*
	cap drop __temp_reghdfe_resid__
	cap matrix drop reghdfe_statsmatrix
	if (`rc' == 132) {
		di as text "- If you got the {it:parentheses unbalanced} error, note that IV/2SLS was moved to {help ivreghdfe}"
		di as smcl `"- Latest version: {browse "https://github.com/sergiocorreia/ivreghdfe":https://github.com/sergiocorreia/ivreghdfe}"'
		di as smcl `"- SSC version: {stata "net describe ivreghdfe, from(http://fmwww.bc.edu/RePEc/bocode/i)"}"'
		di as smcl `"- Note: the older functionality can still be accessed through the {it:old} option"'
	}
	if (`rc') exit `rc'
end

program Parse
	* Trim whitespace (caused by "///" line continuations; aesthetic only)
	mata: st_local("0", stritrim(st_local("0")))

	* Main syntax
	#d;
	syntax varlist(fv ts numeric) [if] [in] [aw pw fw/] , [

		/* Model */
		Absorb(string) NOAbsorb
		SUmmarize SUmmarize2(string asis) /* simulate implicit options */

		/* Standard Errors */
		VCE(string) CLuster(string)

		/* Diagnostic */
		Verbose(numlist min=1 max=1 >=-1 <=5 integer)
		TIMEit

		/* Speedup and memory Tricks */
		NOSAMPle /* do not save e(sample) */
		COMPACT /* use as little memory as possible but is slower */

		/* Extra display options (based on regress) */
		noHEader noTABle noFOOTnote

		/* Undocumented */
		KEEPSINgletons
		OLD /* use latest v3 */
		NOTES(string) /* NOTES(key=value ...), will be stored on e() */

		] [*] /* capture optimization options, display options, etc. */
		;
	#d cr

	* Unused
	* SAVEcache
	* USEcache
	* CLEARcache

	* Convert options to boolean
	if ("`verbose'" == "") loc verbose 0
	loc timeit = ("`timeit'"!="")
	loc drop_singletons = ("`keepsingletons'" == "")
	loc compact = ("`compact'" != "")

	if (`timeit') timer on 29

	* Sanity checks
	if (`verbose'>-1 & "`keepsingletons'"!="") {
		loc url "http://scorreia.com/reghdfe/nested_within_cluster.pdf"
		loc msg "WARNING: Singleton observations not dropped; statistical significance is biased"
		di as error `"`msg' {browse "`url'":(link)}"'
	}
	if ("`cluster'"!="") {
		_assert ("`vce'"==""), msg("cannot specify both cluster() and vce()")
		loc vce cluster `cluster'
		loc cluster // clear it to avoid bugs in subsequent lines
	}

	* Split varlist into <depvar> and <indepvars>
	ms_parse_varlist `varlist'
	if (`verbose' > 0) {
		di as text _n "## Parsing varlist: {res}`varlist'"
		return list
	}
	loc depvar `r(depvar)'
	loc indepvars `r(indepvars)'
	loc fe_format "`r(fe_format)'"
	loc basevars `r(basevars)'

	* Parse Weights
	if ("`weight'"!="") {
		unab exp : `exp', min(1) max(1) // simple weights only
	}

	* Parse VCE
	ms_parse_vce, vce(`vce') weighttype(`weight')
	if (`verbose' > 0) {
		di as text _n "## Parsing vce({res}`vce'{txt})"
		sreturn list
	}
	loc vcetype = "`s(vcetype)'"
	loc num_clusters = `s(num_clusters)'
	loc clustervars = "`s(clustervars)'"
	loc base_clustervars = "`s(base_clustervars)'"
	loc vceextra = "`s(vceextra)'"

	* Select sample (except for absvars)
	loc varlist `depvar' `indepvars' `base_clustervars'
	tempvar touse
	marksample touse, strok // based on varlist + cluster + if + in + weight

	* Parse noabsorb
	_assert  ("`absorb'`noabsorb'" != ""), msg("option {bf:absorb()} or {bf:noabsorb} required")
	if ("`noabsorb'" != "") {
		_assert ("`absorb'" == ""), msg("{bf:absorb()} and {bf:noabsorb} are mutually exclusive")
	}

	if (`timeit') timer off 29

	* Construct HDFE object
	// SYNTAX: fixed_effects(absvars | , touse, wtype, wtvar, dropsing, verbose)
	ms_add_comma, loc(absorb) cmd(`"`absorb'"') opt(`"`options'"')
	if (`timeit') timer on 20
	mata: HDFE = fixed_effects(`"`absorb'"', "`touse'", "`weight'", "`exp'", `drop_singletons', `verbose')
	if (`timeit') timer off 20
	mata: HDFE.cmdline = "reghdfe " + st_local("0")
	loc options `s(options)'

	mata: st_local("N", strofreal(HDFE.N))
	if (`N' == 0) error 2000

	* Fill out HDFE object
	* mata: HDFE.varlist = "`base_varlist'"
	mata: HDFE.depvar = "`depvar'"
	mata: HDFE.indepvars = "`indepvars'"
	mata: HDFE.vcetype = "`vcetype'"
	mata: HDFE.num_clusters = `num_clusters'
	mata: HDFE.clustervars = tokens("`clustervars'")
	mata: HDFE.base_clustervars = tokens("`base_clustervars'")
	mata: HDFE.vceextra = "`vceextra'"

	* Preserve memory
	mata: HDFE.compact = `compact'
	if (`compact') {
		loc panelvar "`_dta[_TSpanel]'"
		loc timevar "`_dta[_TStvar]'"

		cap conf var `panelvar', exact
		if (c(rc)) loc panelvar

		cap conf var `timevar', exact
		if (c(rc)) loc timevar

		mata: HDFE.panelvar = "`panelvar'"
		mata: HDFE.timevar = "`timevar'"
		c_local keepvars `basevars' `base_clustervars' `panelvar' `timevar' // `exp'
	}

	* Parse summarize
	if ("`summarize'" != "") {
		_assert ("`summarize2'" == ""), msg("summarize() syntax error")
		loc summarize2 mean min max  // default values
	}
	ParseSummarize `summarize2'
	mata: HDFE.summarize_stats = "`s(stats)'"
	mata: HDFE.summarize_quietly = `s(quietly)'

	* Parse misc options
	mata: HDFE.notes = `"`notes'"'
	mata: HDFE.store_sample = ("`nosample'"=="")
	mata: HDFE.timeit = `timeit'

	* Parse Coef Table Options (do this last!)
	_get_diopts diopts options, `options' // store in `diopts', and the rest back to `options'
	loc diopts `diopts' `header' `table' `footnote'
	_assert (`"`options'"'==""), msg(`"invalid options: `options'"')
	if ("`hascons'"!="") di in ye "(option ignored: `hascons')"
	if ("`tsscons'"!="") di in ye "(option ignored: `tsscons')"
	mata: HDFE.diopts = `"`diopts'"'
end

program ParseSummarize, sclass
	sreturn clear
	syntax [namelist(name=stats)] , [QUIetly]
	local quietly = ("`quietly'"!="")
	sreturn loc stats "`stats'"
	sreturn loc quietly = `quietly'
end

// --------------------------------------------------------------------------

program Estimate, eclass
	ereturn clear

	* Parse and fill out HDFE object
	Parse `0'
	mata: st_local("timeit", strofreal(HDFE.timeit))
	mata: st_local("compact", strofreal(HDFE.compact))
	mata: st_local("verbose", strofreal(HDFE.verbose))

	* Compute degrees-of-freedom
	if (`timeit') timer on 21
	mata: HDFE.estimate_dof()
	if (`timeit') timer off 21

	* Save updated e(sample) (singletons reduce sample);
	* required to parse factor variables to partial out
	if (`timeit') timer on 29
	tempvar touse
	mata: HDFE.save_touse("`touse'")
	if (`timeit') timer off 29

	* Expand varlists
	if (`timeit') timer on 22
	mata: st_local("depvar", HDFE.depvar)
	mata: st_local("indepvars", HDFE.indepvars)
	if (`verbose' > 0) di as text _n "## Parsing and expanding indepvars: {res}`indepvars'"
	ms_expand_varlist `indepvars' if `touse'
	if (`verbose' > 0) return list
	mata: HDFE.fullindepvars = "`r(fullvarlist)'"
	mata: HDFE.indepvars = "`r(varlist)'"
	mata: HDFE.not_basevar = strtoreal(tokens("`r(not_omitted)'"))
	mata: HDFE.varlist = "`depvar' `r(varlist)'"
	if (`timeit') timer off 22

	* Stats
	mata: st_local("stats", HDFE.summarize_stats)
	if ("`stats'" != "") Stats `touse'

	* Condition number
	mata: HDFE.estimate_cond()

	* Preserve
	if (`compact') {
		if (`verbose' > 0) di as text "## Preserving dataset"
		preserve
		novarabbrev keep `keepvars'
	}

	* Partial out; save TSS of depvar
	if (`timeit') timer on 23
	// SYNTAX: partial_out(Varlist/Matrix | , Save TSS if HDFE.tss is missing? [0], Standardize data? [1], First col is depvar? [1])
	// Note: standardize=2 will standardize, partial out, and return the data standardized!
	mata: hdfe_variables = HDFE.partial_out(HDFE.varlist, 1, 2, .)
	if (`timeit') timer off 23

	* Regress
	if (`timeit') timer on 24
	tempname b V N rank df_r
	mata: reghdfe_post_ols(HDFE, hdfe_variables, "`b'", "`V'", "`N'", "`rank'", "`df_r'")
	mata: hdfe_variables = .
	* Restore
	if (`compact') {
		if (`verbose' > 0) di as text "## Restoring dataset"
		restore
		mata: st_local("residuals", HDFE.residuals)
		if ("`residuals'" != "") mata: HDFE.save_variable(HDFE.residuals, HDFE.residuals_vector, "Residuals")
	}
	RegressOLS `touse' `b' `V' `N' `rank' `df_r'
	if (`timeit') timer off 24

	* (optional) Store FEs
	if (`timeit') timer on 29
	reghdfe, store_alphas
	if (`timeit') timer off 29

	* View estimation tables
	mata: st_local("diopts", HDFE.diopts)
	Replay, `diopts'

	if (`timeit') {
		di as text _n "{bf: Timer results:}"
		timer list
		di as text "Legend: 20: Create HDFE object; 21: Estimate DoF; 22: expand varlists; 23: partial out; 24: regress; 29: rest"
		di
	}
end

program RegressOLS, eclass
	args touse b V N rank df_r

	mata: st_local("store_sample", strofreal(HDFE.store_sample))
	if (`store_sample') loc esample "esample(`touse')"

	mata: st_local("indepvars", HDFE.fullindepvars)
	if ("`indepvars'" != "") {
		matrix colnames `b' = `indepvars'
		matrix colnames `V' = `indepvars'
		matrix rownames `V' = `indepvars'
		_ms_findomitted `b' `V'
		ereturn post `b' `V', `esample' buildfvinfo depname(`depvar')
	}
	else {
		ereturn post, `esample' buildfvinfo depname(`depvar')
	}

	ereturn scalar N       = `N'
	ereturn scalar rank    = `rank'
	ereturn scalar df_r    = `df_r'
	ereturn local  cmd     "reghdfe"
	mata: HDFE.post()

	* Post stats
	cap conf matrix reghdfe_statsmatrix
	if (!c(rc)) {
		ereturn matrix summarize = reghdfe_statsmatrix
		mata: st_local("summarize_quietly", strofreal(HDFE.summarize_quietly))
		ereturn scalar summarize_quietly = `summarize_quietly'
	}
end

program Replay, rclass
	syntax [, noHEader noTABle noFOOTnote *]

	if `"`e(cmd)'"' != "reghdfe"  {
	        error 301
	}

	_get_diopts options, `options'
	if ("`header'" == "") {
		reghdfe_header // _coef_table_header
		di ""
	}
	if ("`table'" == "") {
		_coef_table, `options' // ereturn display, `options'
		return add // adds r(level), r(table), etc. to ereturn (before the footnote deletes them)
	}
	if ("`footnote'" == "") {
		reghdfe_footnote
	}

	* Replay stats
	if (e(summarize_quietly)==0) {
		di as text _n "{sf:Regression Summary Statistics:}" _c
		matlist e(summarize)', border(top bottom) rowtitle(Variable) // twidth(18)
	}
end

program Stats
	args touse
	* Optional weights
	mata: st_local("weight", sprintf("[%s=%s]", HDFE.weight_type, HDFE.weight_var))
	assert "`weight'" != ""
	if ("`weight'" == "[=]") loc weight
	loc weight : subinstr local weight "[pweight" "[aweight"

	mata: st_local("stats", HDFE.summarize_stats)
	mata: st_local("varlist", HDFE.varlist)
	mata: st_local("cvars", invtokens(HDFE.cvars))
	loc full_varlist `varlist' `cvars'

	* quick workaround b/c -tabstat- does not support factor variables
	fvrevar `full_varlist', list
	loc full_varlist `r(varlist)'

	qui tabstat `full_varlist' if `touse' `weight' , stat(`stats') col(stat) save
	matrix reghdfe_statsmatrix = r(StatTotal)
end

findfile "reghdfe.mata"
include "`r(fn)'"

exit

*! version 5.8.0 27dec2019

program reghdfe, eclass
	* Intercept old+version and cache(clear) operation
	cap syntax, old [version CACHE(passthru)]
	if !c(rc) {
        	reghdfe_old, `version' `cache'
		exit
	}

	* Intercept old
	cap syntax anything(everything) [fw aw pw/], [*] old
	if !c(rc) {
		di as error "(running historical version of reghdfe)"
		if ("`weight'"!="") local weightexp [`weight'=`exp']
		reghdfe_old `anything' `weightexp', `options'
		exit
	}

	* Aux. subcommands
	cap syntax, [*]
	if inlist("`options'", "check", "compile", "reload", "update", "version", "requirements", "store_alphas") {
		if ("`options'"=="compile") loc args force
		if ("`options'"=="check") loc options compile
		if ("`options'"=="update") {
			loc args 1
			loc options reload
		}
		loc subcmd = proper("`options'")
		`subcmd' `args'
	}
	else if replay() {
		Replay `0'
	}
	else {
		Cleanup 0
		ms_get_version ftools, min_version("2.36.1") // Compile // takes 0.01s to run this useful check (ensures .mlib exists)
		cap noi Estimate `0'
		Cleanup `c(rc)'
	}
end

program Compile
	args force

	* Check dependencies
	ftools, check // in case lftools.mlib does not exist or is outdated
	ms_get_version ftools, min_version("2.34.0")
	ms_get_version reghdfe // save local package_version
	loc list_objects "FixedEffects() fixed_effects() BipartiteGraph()"
	loc list_functions "reghdfe_*() transform_*() accelerate_*() panelmean() panelsolve_*() lsmr()"
	loc list_misc "weighted_quadcolsum() safe_divide() check_convergence() precompute_inv_xx() _st_data_wrapper()"
	// TODO: prefix everything with reghdfe_*

	ms_compile_mata, ///
		package(reghdfe) ///
		version(`package_version') ///
		fun("`list_objects' `list_functions' `list_misc'") ///
		verbose ///
		`force'
end

program Reload
	* Internal debugging tool.
	* Updates dependencies and reghdfe from local path or from github
	* Usage:
	* 	reghdfe, update // from c:\git\..
	* 	reghdfe, reload // from github

	args online
	if ("`online'" == "") loc online 0

	di as text _n "{bf:reghdfe: updating required packages}"
	di as text "{hline 64}"

	* -ftools- https://github.com/sergiocorreia/ftools/
	cap ado uninstall ftools
	if (`online') net install ftools, from("https://github.com/sergiocorreia/ftools/raw/master/src/")
	if (!`online') net install ftools, from("c:\git\ftools\src")
	di as text "{hline 64}"
	ftools, compile
	di as text "{hline 64}"

	* Update -reghdfe-
	di as text _n  _n "{bf:reghdfe: updating self}"
	di as text "{hline 64}"
	qui ado uninstall reghdfe
	if (`online') net install reghdfe, from("https://github.com/sergiocorreia/reghdfe/raw/master/src/")
	if (!`online') net install reghdfe, from("c:\git\reghdfe\src")
	qui which reghdfe
	di as text "{hline 64}"
	reghdfe, compile
	di as text "{hline 64}"

	* Cleaning up
	di as text _n "{bf:Note:} You need to run {stata program drop _all} now."
end

program Version
	which reghdfe
	Requirements
end

program Requirements
	di as text _n "Required packages installed?"
	loc reqs ftools
	// ivreg2 avar tuples group3hdfe
	if (c(stata_version)<13) loc reqs `reqs' boottest

	loc ftools_github "https://github.com/sergiocorreia/ftools/raw/master/src/"

	loc error 0

	foreach req of local reqs {
		loc fn `req'.ado
		cap findfile `fn'
		if (_rc) {
			loc error 1
			di as text "{lalign 20:- `req'}" as error "not" _c
			di as text "    {stata ssc install `req':install from SSC}" _c
			if inlist("`req'", "ftools") {
				loc github ``req'_github'
				di as text `"    {stata `"net install `req', from(`"`github'"')"':install from github}"'
			}
			else {
				di as text // newline
			}
		}
		else {
			di as text "{lalign 20:- `req'}" as text "yes"
		}
	}

	if (`error') exit 601
end

program Store_Alphas, eclass
	mata: st_local("save_any_fe", strofreal(HDFE.save_any_fe))
	assert inlist(`save_any_fe', 0, 1)
	if (`save_any_fe') {
		_assert e(depvar) != "", msg("e(depvar) is empty")
		_assert e(resid) != "", msg("e(resid) is empty")
		// we can't use -confirm var- because it might have TS operators
		fvrevar `e(depvar)', list
		confirm numeric var `e(resid)', exact
		tempvar d
		if (e(rank)) {
			qui _predict double `d' if e(sample), xb
		}
		else if (e(report_constant)) {
			gen double `d' = _b[_cons] if e(sample)
		}
		else {
			gen double `d' = 0 if e(sample)
		}
		qui replace `d' = `e(depvar)' - `d' - `e(resid)' if e(sample)

		mata: HDFE.store_alphas("`d'")
		drop `d'

		// Drop resid if we don't want to save it; and update e(resid)
		cap drop __temp_reghdfe_resid__
		if (!c(rc)) ereturn local resid
	}
end

program Cleanup
	args rc
	cap mata: mata drop HDFE
	cap mata: mata drop hdfe_*
	cap drop __temp_reghdfe_resid__
	cap matrix drop reghdfe_statsmatrix
	if (`rc' == 132) {
		di as text "- If you got the {it:parentheses unbalanced} error, note that IV/2SLS was moved to {help ivreghdfe}"
		di as smcl `"- Latest version: {browse "https://github.com/sergiocorreia/ivreghdfe":https://github.com/sergiocorreia/ivreghdfe}"'
		di as smcl `"- SSC version: {stata "net describe ivreghdfe, from(http://fmwww.bc.edu/RePEc/bocode/i)"}"'
		di as smcl `"- Note: the older functionality can still be accessed through the {it:old} option"'
	}
	if (`rc') exit `rc'
end

program Parse
	* Trim whitespace (caused by "///" line continuations; aesthetic only)
	mata: st_local("0", stritrim(st_local("0")))

	* Main syntax
	#d;
	syntax varlist(fv ts numeric) [if] [in] [aw pw fw/] , [

		/* Model */
		Absorb(string) NOAbsorb
		SUmmarize SUmmarize2(string asis) /* simulate implicit options */

		/* Standard Errors */
		VCE(string) CLuster(string)

		/* Diagnostic */
		Verbose(numlist min=1 max=1 >=-1 <=5 integer)
		TIMEit

		/* Speedup and memory Tricks */
		NOSAMPle /* do not save e(sample) */
		COMPACT /* use as little memory as possible but is slower */

		/* Extra display options (based on regress) */
		noHEader noTABle noFOOTnote

		/* Undocumented */
		KEEPSINgletons
		OLD /* use latest v3 */
		NOTES(string) /* NOTES(key=value ...), will be stored on e() */

		] [*] /* capture optimization options, display options, etc. */
		;
	#d cr

	* Unused
	* SAVEcache
	* USEcache
	* CLEARcache

	* Convert options to boolean
	if ("`verbose'" == "") loc verbose 0
	loc timeit = ("`timeit'"!="")
	loc drop_singletons = ("`keepsingletons'" == "")
	loc compact = ("`compact'" != "")

	if (`timeit') timer on 29

	* Sanity checks
	if (`verbose'>-1 & "`keepsingletons'"!="") {
		loc url "http://scorreia.com/reghdfe/nested_within_cluster.pdf"
		loc msg "WARNING: Singleton observations not dropped; statistical significance is biased"
		di as error `"`msg' {browse "`url'":(link)}"'
	}
	if ("`cluster'"!="") {
		_assert ("`vce'"==""), msg("cannot specify both cluster() and vce()")
		loc vce cluster `cluster'
		loc cluster // clear it to avoid bugs in subsequent lines
	}

	* Split varlist into <depvar> and <indepvars>
	ms_parse_varlist `varlist'
	if (`verbose' > 0) {
		di as text _n "## Parsing varlist: {res}`varlist'"
		return list
	}
	loc depvar `r(depvar)'
	loc indepvars `r(indepvars)'
	loc fe_format "`r(fe_format)'"
	loc basevars `r(basevars)'

	* Parse Weights
	if ("`weight'"!="") {
		unab exp : `exp', min(1) max(1) // simple weights only
	}

	* Parse VCE
	ms_parse_vce, vce(`vce') weighttype(`weight')
	if (`verbose' > 0) {
		di as text _n "## Parsing vce({res}`vce'{txt})"
		sreturn list
	}
	loc vcetype = "`s(vcetype)'"
	loc num_clusters = `s(num_clusters)'
	loc clustervars = "`s(clustervars)'"
	loc base_clustervars = "`s(base_clustervars)'"
	loc vceextra = "`s(vceextra)'"

	* Select sample (except for absvars)
	loc varlist `depvar' `indepvars' `base_clustervars'
	tempvar touse
	marksample touse, strok // based on varlist + cluster + if + in + weight

	* Parse noabsorb
	_assert  ("`absorb'`noabsorb'" != ""), msg("option {bf:absorb()} or {bf:noabsorb} required")
	if ("`noabsorb'" != "") {
		_assert ("`absorb'" == ""), msg("{bf:absorb()} and {bf:noabsorb} are mutually exclusive")
	}

	if (`timeit') timer off 29

	* Construct HDFE object
	// SYNTAX: fixed_effects(absvars | , touse, wtype, wtvar, dropsing, verbose)
	ms_add_comma, loc(absorb) cmd(`"`absorb'"') opt(`"`options'"')
	if (`timeit') timer on 20
	mata: HDFE = fixed_effects(`"`absorb'"', "`touse'", "`weight'", "`exp'", `drop_singletons', `verbose')
	if (`timeit') timer off 20
	mata: HDFE.cmdline = "reghdfe " + st_local("0")
	loc options `s(options)'

	mata: st_local("N", strofreal(HDFE.N))
	if (`N' == 0) error 2000

	* Fill out HDFE object
	* mata: HDFE.varlist = "`base_varlist'"
	mata: HDFE.depvar = "`depvar'"
	mata: HDFE.indepvars = "`indepvars'"
	mata: HDFE.vcetype = "`vcetype'"
	mata: HDFE.num_clusters = `num_clusters'
	mata: HDFE.clustervars = tokens("`clustervars'")
	mata: HDFE.base_clustervars = tokens("`base_clustervars'")
	mata: HDFE.vceextra = "`vceextra'"

	* Preserve memory
	mata: HDFE.compact = `compact'
	if (`compact') {
		loc panelvar "`_dta[_TSpanel]'"
		loc timevar "`_dta[_TStvar]'"

		cap conf var `panelvar', exact
		if (c(rc)) loc panelvar

		cap conf var `timevar', exact
		if (c(rc)) loc timevar

		mata: HDFE.panelvar = "`panelvar'"
		mata: HDFE.timevar = "`timevar'"
		c_local keepvars `basevars' `base_clustervars' `panelvar' `timevar' // `exp'
	}

	* Parse summarize
	if ("`summarize'" != "") {
		_assert ("`summarize2'" == ""), msg("summarize() syntax error")
		loc summarize2 mean min max  // default values
	}
	ParseSummarize `summarize2'
	mata: HDFE.summarize_stats = "`s(stats)'"
	mata: HDFE.summarize_quietly = `s(quietly)'

	* Parse misc options
	mata: HDFE.notes = `"`notes'"'
	mata: HDFE.store_sample = ("`nosample'"=="")
	mata: HDFE.timeit = `timeit'

	* Parse Coef Table Options (do this last!)
	_get_diopts diopts options, `options' // store in `diopts', and the rest back to `options'
	loc diopts `diopts' `header' `table' `footnote'
	_assert (`"`options'"'==""), msg(`"invalid options: `options'"')
	if ("`hascons'"!="") di in ye "(option ignored: `hascons')"
	if ("`tsscons'"!="") di in ye "(option ignored: `tsscons')"
	mata: HDFE.diopts = `"`diopts'"'
end

program ParseSummarize, sclass
	sreturn clear
	syntax [namelist(name=stats)] , [QUIetly]
	local quietly = ("`quietly'"!="")
	sreturn loc stats "`stats'"
	sreturn loc quietly = `quietly'
end

// --------------------------------------------------------------------------

program Estimate, eclass
	ereturn clear

	* Parse and fill out HDFE object
	Parse `0'
	mata: st_local("timeit", strofreal(HDFE.timeit))
	mata: st_local("compact", strofreal(HDFE.compact))
	mata: st_local("verbose", strofreal(HDFE.verbose))

	* Compute degrees-of-freedom
	if (`timeit') timer on 21
	mata: HDFE.estimate_dof()
	if (`timeit') timer off 21

	* Save updated e(sample) (singletons reduce sample);
	* required to parse factor variables to partial out
	if (`timeit') timer on 29
	tempvar touse
	mata: HDFE.save_touse("`touse'")
	if (`timeit') timer off 29

	* Expand varlists
	if (`timeit') timer on 22
	mata: st_local("depvar", HDFE.depvar)
	mata: st_local("indepvars", HDFE.indepvars)
	if (`verbose' > 0) di as text _n "## Parsing and expanding indepvars: {res}`indepvars'"
	ms_expand_varlist `indepvars' if `touse'
	if (`verbose' > 0) return list
	mata: HDFE.fullindepvars = "`r(fullvarlist)'"
	mata: HDFE.indepvars = "`r(varlist)'"
	mata: HDFE.not_basevar = strtoreal(tokens("`r(not_omitted)'"))
	mata: HDFE.varlist = "`depvar' `r(varlist)'"
	if (`timeit') timer off 22

	* Stats
	mata: st_local("stats", HDFE.summarize_stats)
	if ("`stats'" != "") Stats `touse'

	* Condition number
	mata: HDFE.estimate_cond()

	* Preserve
	if (`compact') {
		if (`verbose' > 0) di as text "## Preserving dataset"
		preserve
		novarabbrev keep `keepvars'
	}

	* Partial out; save TSS of depvar
	if (`timeit') timer on 23
	// SYNTAX: partial_out(Varlist/Matrix | , Save TSS if HDFE.tss is missing? [0], Standardize data? [1], First col is depvar? [1])
	// Note: standardize=2 will standardize, partial out, and return the data standardized!
	mata: hdfe_variables = HDFE.partial_out(HDFE.varlist, 1, 2, .)
	if (`timeit') timer off 23

	* Regress
	if (`timeit') timer on 24
	tempname b V N rank df_r
	mata: reghdfe_post_ols(HDFE, hdfe_variables, "`b'", "`V'", "`N'", "`rank'", "`df_r'")
	mata: hdfe_variables = .
	* Restore
	if (`compact') {
		if (`verbose' > 0) di as text "## Restoring dataset"
		restore
		mata: st_local("residuals", HDFE.residuals)
		if ("`residuals'" != "") mata: HDFE.save_variable(HDFE.residuals, HDFE.residuals_vector, "Residuals")
	}
	RegressOLS `touse' `b' `V' `N' `rank' `df_r'
	if (`timeit') timer off 24

	* (optional) Store FEs
	if (`timeit') timer on 29
	reghdfe, store_alphas
	if (`timeit') timer off 29

	* View estimation tables
	mata: st_local("diopts", HDFE.diopts)
	Replay, `diopts'

	if (`timeit') {
		di as text _n "{bf: Timer results:}"
		timer list
		di as text "Legend: 20: Create HDFE object; 21: Estimate DoF; 22: expand varlists; 23: partial out; 24: regress; 29: rest"
		di
	}
end

program RegressOLS, eclass
	args touse b V N rank df_r

	mata: st_local("store_sample", strofreal(HDFE.store_sample))
	if (`store_sample') loc esample "esample(`touse')"

	mata: st_local("indepvars", HDFE.fullindepvars)
	if ("`indepvars'" != "") {
		matrix colnames `b' = `indepvars'
		matrix colnames `V' = `indepvars'
		matrix rownames `V' = `indepvars'
		_ms_findomitted `b' `V'
		ereturn post `b' `V', `esample' buildfvinfo depname(`depvar')
	}
	else {
		ereturn post, `esample' buildfvinfo depname(`depvar')
	}

	ereturn scalar N       = `N'
	ereturn scalar rank    = `rank'
	ereturn scalar df_r    = `df_r'
	ereturn local  cmd     "reghdfe"
	mata: HDFE.post()

	* Post stats
	cap conf matrix reghdfe_statsmatrix
	if (!c(rc)) {
		ereturn matrix summarize = reghdfe_statsmatrix
		mata: st_local("summarize_quietly", strofreal(HDFE.summarize_quietly))
		ereturn scalar summarize_quietly = `summarize_quietly'
	}
end

program Replay, rclass
	syntax [, noHEader noTABle noFOOTnote *]

	if `"`e(cmd)'"' != "reghdfe"  {
	        error 301
	}

	_get_diopts options, `options'
	if ("`header'" == "") {
		reghdfe_header // _coef_table_header
		di ""
	}
	if ("`table'" == "") {
		_coef_table, `options' // ereturn display, `options'
		return add // adds r(level), r(table), etc. to ereturn (before the footnote deletes them)
	}
	if ("`footnote'" == "") {
		reghdfe_footnote
	}

	* Replay stats
	if (e(summarize_quietly)==0) {
		di as text _n "{sf:Regression Summary Statistics:}" _c
		matlist e(summarize)', border(top bottom) rowtitle(Variable) // twidth(18)
	}
end

program Stats
	args touse
	* Optional weights
	mata: st_local("weight", sprintf("[%s=%s]", HDFE.weight_type, HDFE.weight_var))
	assert "`weight'" != ""
	if ("`weight'" == "[=]") loc weight
	loc weight : subinstr local weight "[pweight" "[aweight"

	mata: st_local("stats", HDFE.summarize_stats)
	mata: st_local("varlist", HDFE.varlist)
	mata: st_local("cvars", invtokens(HDFE.cvars))
	loc full_varlist `varlist' `cvars'

	* quick workaround b/c -tabstat- does not support factor variables
	fvrevar `full_varlist', list
	loc full_varlist `r(varlist)'

	qui tabstat `full_varlist' if `touse' `weight' , stat(`stats') col(stat) save
	matrix reghdfe_statsmatrix = r(StatTotal)
end

findfile "reghdfe.mata"
include "`r(fn)'"

exit


iegrpah : help expanded examples

iegrpah : help expanded examples

{smcl}
{* 26 Dec 2016}{...}
{hline}
help for {hi:iegraph}
{hline}

{title:Title}

{phang2}{cmdab:iegraph} {hline 2} Generates graphs based on regressions with treatment dummies common in impact evaluations.

{title:Syntax}

{phang2}
{cmdab:iegraph} {varlist}
, [{cmd:noconfbars} {cmdab:ti:tle(}{it:string}{cmd:)}
{cmdab:save(}{it:string}{cmd:)} {cmdab:confbarsnone(}{it:varlist}{cmd:)}
{it:any towway graph options}
]

{marker opts}{...}
{synoptset 22}{...}
{synopthdr:options}
{synoptline}
{synopt :{cmd:noconfbars}} Removes the confidence interval bars from graphs for all treatments.{p_end}
{synopt :{cmdab:ti:tle(}{it:string}{cmd:)}} Manually sets the title of the graph.{p_end}
{synopt :{cmdab:save(}{it:string}{cmd:)}} Sets the filename and the directory to which the graph will be set.{p_end}
{synopt :{cmdab:confbarsnone(}{it:varlist}{cmd:)}} Removes confidence interval bars from only the {it:varlist} listed.{p_end}
{synopt :{cmdab:yzero}} Forces y-axis on the graph to start at 0.{p_end}
{synopt :{cmdab: Graph Options}} Options that can be used with normal graph commands can also be used. See example 2 for details.{p_end}
{synoptline}

{marker desc}
{title:Description}

{pstd}{cmdab:iegraph} This command creates bar graphs on the based on the coefficients
	of treatment dummies in regression results. This command is developed for reading
	stored results of from two types of impact evaluation regression models, but there
	are countless of other examples where the command also can be used. {cmd:iegraph} must
	be used immedeatly after running the regression or as long as teh regression result is
	still stored in or restored to Stata's {help ereturn} results.

{pstd}{bf:Model 1: OLS with Treatment Dummies}{break}The most typical impact evaluation regression is
	to have the outcome variable as the dependent varaible and one dummy for each
	treatment arm where control is the omitted category. This regressions can also include
	covariates, fixed effects etc., but as long as the treatment status is defined by
	mutually exclusive dummy variables. See especially example 1 and 2 below. This command
	works with any number of treatment arms but works best from two arms (treatment
	and control) to five treatment arms (4 different treatments and control). More
	arms than that may result in a still correct but perhaps cluttered  graph.

{pstd}{bf:Model 2: Difference-in-Differneces}{break}Another typical regression model in impact
	evaluations are difference-in-difference (Diff-in-Diff) models with two treatment arms (treatment
	and control) and two time periods. If the Diff-in-Diff regression is specified as having the
	outcome variable as the dependent variable and three dummy variables (time, treatment
	and timeXtreatment) as the independent varaibles, then this command will produce a nice
	graph. Controls, treatment effects etc. may be added to the regression model. See especially example 3.

{pstd}{bf:Graph Output}{break}The graph generated by this command is created using the following values. The
	control bar is the mean of the outcome variable for the control group. It is not
	the constant from the regression as those are not identical if, for example, fixed effects
	and covariates were used. For each treatment group the bar is the sum of the value
	of the control bar and the beta coefficient in the regression of the corresponding
	treatment dummy. The confidence intervals are calculated from the variance in the
	beta coefficients in the regression.{p_end}

{pstd}The graph also includes the N for each treatment arm in the regression and uses
	that value as labels on the x-axis. Stars are added to this value if the corresponding
	coefficient is statistically different from zero in the regression{p_end}

{marker optslong}
{title:Options}

{phang}{cmd:noconfbars} Removes the confidence interval bars from graphs for all
	treatments. The default value for the confidence interval bars ar 95%. {p_end}

{phang}{cmdab:ti:tle(}{it:string}{cmd:)} Manually sets the title of the graph.{p_end}

{phang}{cmdab:save(}{it:string}{cmd:)} Sets the filename and the directory to which
	the graph will be set.{p_end}

{phang}{cmdab:confbarsnone(}{it:varlist}{cmd:)} Removes confidence interval bars
	from only the {it:varlist} listed. The remaining variables in the graphs which
	have not been specified in {cmdab:confbarsnone} will still have the confidence
	interval bars. {p_end}

{phang}{cmdab:yzero} Manually sets the y-axis of the graph to start at zero instead of the Stata default.{p_end}

{marker optslong}
{title:Examples}

{pstd} {hi:Example 1.}

{pmore} {inp:regress} {it:outcomevar treatment_dummy}{break}
		{inp:iegraph} {it:treatment_dummy} , {inp:title({it:"Treatment Effect on Outcome"})}

{pmore}In the example above, there is only two treatment arms (treatment and
		control). {it:treatment_dummy} has a 1 for all treatment observations and
		a 0 for all control observations. The graph will have one bar for control and
		it shows the mean for {it:outcomevar} for all observations in control. The
		second bar in the graph will be the sum of that mean and the coefficient
		for {it:treatment_dummy} in the regression. The graph will also have the
		title: Treatment Effect on Outcome.

{pstd} {hi:Example 2.}

{pmore} {inp:regress} {it:income tmt_1 tmt_2 age education}{inp:, cluster(}{it:district}{inp:)}{break}
		{inp:iegraph} {it:tmt_1 tmt_2}{inp:, noconfbars yzero title({it:"Treatment effect on income"}) }

{pmore}In the example above, the treatment effect on income in reserached. There
		are three treatment arms; control, treatment 1 ({it:tmt_1}) and treatment
		2 ({it:tmt_2}). It is important that no observation has the value 1 in
		both {it:tmt_1} and {it:tmt_2} (i.e. no observation is in more than one
		treatment) and some observations must have the value 0 in both {it:tmt_1}
		and {it:tmt_2} (i.e. control observations). The variables {it:age} and
		{it:education} are covariates (control variables) and is not included
		in {cmd:iegraph}. {inp:noconfbars} omitts the confidence intervall bars
		, and {inp:yzero} sets the y-axis to start at 0.

{pstd} {hi:Example 3.}

{pmore} {inp:regress} {it:chld_wght time treat timeXtreat}{break}
		{inp:iegraph} {it:time treat timeXtreat} {inp:, title({it:"Treatment effect on Child Weight (Diff-in-Diff)"})}

{pmore}In the example above, the data set is a panel data set with two time
		periods and the regression estimates the treatment effect on child weight
		using a Difference-in-Differences model.
		The dummy variable {it:time} indicates if it is time period 0 or 1.
		The dummy variable {it:treat} indicates if the observation is treatment
		or control. {it:timeXtreat} is the interaction term of {it:time}
		and {it:treat}. This the standard way to set up a Difference-in-Differences
		regression model.

{pstd} {hi:Example 4.}

{pmore} {inp:regress} {it:harvest T1 T2 T3 } {break}
		{inp:iegraph} {it:T1 T2 T3} {inp:, title({it:"Treatment effect on harvest"})
		xlabel(,angle(45)) yzero ylabel(minmax) save({it:"$Output/Graph1.gph"})}

{pmore}The example above shows how to save a graph to disk. It also shows that
	most two-way graph options can be used. In this example the {cmd:iegraph}
	option {cdm:yzero} conflicts with the two-way option {cmd:ylabel(minmax)}.
	In such a case the user specified option takes presidence over {cmd:iegraph}
	options like {cdm:yzero}.

{title:Acknowledgements}

{phang}We would like to acknowledge the help in testing and proofreading we received in relation to this command and help file from (in alphabetic order):{p_end}
{pmore}{break}

{title:Author}

{phang}Kristoffer Bjarkefur & Mrijan Rimal, The World Bank, DECIE

{phang}Please send bug-reports, suggestions and requests for clarifications
		 writing "ietools iegraph" in the subject line to:{break}
		 kbjarkefur@worldbank.org

{phang}You can also see the code, make comments to the code, see the version
		 history of the code, and submit additions or edits to the code through
		 the github repository of ietoolkit:{break}
		 {browse "https://github.com/worldbank/ietoolkit"}

{smcl}
{* 26 Dec 2016}{...}
{hline}
help for {hi:iegraph}
{hline}

{title:Title}

{phang2}{cmdab:iegraph} {hline 2} Generates graphs based on regressions with treatment dummies common in impact evaluations.

{title:Syntax}

{phang2}
{cmdab:iegraph} {varlist}
, [{cmd:noconfbars} {cmdab:ti:tle(}{it:string}{cmd:)}
{cmdab:save(}{it:string}{cmd:)} {cmdab:confbarsnone(}{it:varlist}{cmd:)}
{it:any towway graph options}
]

{marker opts}{...}
{synoptset 22}{...}
{synopthdr:options}
{synoptline}
{synopt :{cmd:noconfbars}} Removes the confidence interval bars from graphs for all treatments.{p_end}
{synopt :{cmdab:ti:tle(}{it:string}{cmd:)}} Manually sets the title of the graph.{p_end}
{synopt :{cmdab:save(}{it:string}{cmd:)}} Sets the filename and the directory to which the graph will be set.{p_end}
{synopt :{cmdab:confbarsnone(}{it:varlist}{cmd:)}} Removes confidence interval bars from only the {it:varlist} listed.{p_end}
{synopt :{cmdab:yzero}} Forces y-axis on the graph to start at 0.{p_end}
{synopt :{cmdab: Graph Options}} Options that can be used with normal graph commands can also be used. See example 2 for details.{p_end}
{synoptline}

{marker desc}
{title:Description}

{pstd}{cmdab:iegraph} This command creates bar graphs on the based on the coefficients
	of treatment dummies in regression results. This command is developed for reading
	stored results of from two types of impact evaluation regression models, but there
	are countless of other examples where the command also can be used. {cmd:iegraph} must
	be used immedeatly after running the regression or as long as teh regression result is
	still stored in or restored to Stata's {help ereturn} results.

{pstd}{bf:Model 1: OLS with Treatment Dummies}{break}The most typical impact evaluation regression is
	to have the outcome variable as the dependent varaible and one dummy for each
	treatment arm where control is the omitted category. This regressions can also include
	covariates, fixed effects etc., but as long as the treatment status is defined by
	mutually exclusive dummy variables. See especially example 1 and 2 below. This command
	works with any number of treatment arms but works best from two arms (treatment
	and control) to five treatment arms (4 different treatments and control). More
	arms than that may result in a still correct but perhaps cluttered  graph.

{pstd}{bf:Model 2: Difference-in-Differneces}{break}Another typical regression model in impact
	evaluations are difference-in-difference (DD) models with two treatment arms (treatment
	and control) and two time periods. If the DD regression is specified as having the
	outcome variable as the dependent variable and three dummy variables (time, treatment
	and timeXtreatment) as the independent varaibles, then this command will produce a nice
	graph. Controls, treatment effects etc. may be added to the regression model. See especially example 3.

{pstd}{bf:Graph Output}{break}The graph generated by this command is created using the following values. The
	control bar is the mean of the outcome variable for the control group. It is not
	the constant from the regression as those are not identical if, for example, fixed effects
	and covariates were used. For each treatment group the bar is the sum of the value
	of the control bar and the beta coefficient in the regression of the corresponding
	treatment dummy. The confidence intervals are calculated from the variance in the
	beta coefficients in the regression.{p_end}

{pstd}The graph also includes the N for each treatment arm in the regression and uses
	that value as labels on the x-axis. Stars are added to this value if the corresponding
	coefficient is statistically different from zero in the regression{p_end}

{marker optslong}
{title:Options}

{phang}{cmd:noconfbars} Removes the confidence interval bars from graphs for all
	treatments. The default value for the confidence interval bars ar 95%. {p_end}

{phang}{cmdab:ti:tle(}{it:string}{cmd:)} Manually sets the title of the graph.{p_end}

{phang}{cmdab:save(}{it:string}{cmd:)} Sets the filename and the directory to which
	the graph will be set.{p_end}

{phang}{cmdab:confbarsnone(}{it:varlist}{cmd:)} Removes confidence interval bars
	from only the {it:varlist} listed. The remaining variables in the graphs which
	have not been specified in {cmdab:confbarsnone} will still have the confidence
	interval bars. {p_end}

{phang}{cmdab:yzero} Manually sets the y-axis of the graph to start at zero instead of the Stata default.{p_end}

{marker optslong}
{title:Examples}

{pstd} {hi:Example 1.}

{pmore} {inp:regress outcomevar treatment_dummy1 treatment_dummy2}{break}
		{inp:iegraph treatment_dummy1 treatment_dummy2 , title({it:string})}

{pmore}In the example above, graphs comparing the effects of {it:treatmentVar1} and
		{it:treatmentVar2} to the mean of the control will be generated. The option
		title({it:string}) gives the graph the title that the user puts in the string.

{pstd} {hi:Example 2.}

{pmore}{inp:iegraph treatment, noconfbars title("Treatment effect on price") save("$Output/Graph1.gph") xlabel(,angle(45)) ylabel(minmax)}

{pmore}In the example above, the observations with value 1 in {it:tmt} will be matched
	towards the nearest, in terms of {it:p_hat}, observations with value 0 in {it:tmt} as
	long as the difference in {it:p_hat} is less than .001. Only observations that has the
	value 1 in variable {it:baseline} will be included in the match.

{title:Acknowledgements}

{phang}I would like to acknowledge the help in testing and proofreading I received in relation to this command and help file from (in alphabetic order):{p_end}
{pmore}{break}

{title:Author}

{phang}Kristoffer Bjarkefur & Mrijan Rimal, The World Bank, DECIE

{phang}Please send bug-reports, suggestions and requests for clarifications
		 writing "ietools iegraph" in the subject line to:{break}
		 kbjarkefur@worldbank.org

{phang}You can also see the code, make comments to the code, see the version
		 history of the code, and submit additions or edits to the code through
		 the github repository of ietoolkit:{break}
		 {browse "https://github.com/worldbank/ietoolkit"}