Skip to content

Commit

Permalink
adding lecture 9
Browse files Browse the repository at this point in the history
  • Loading branch information
@drganghe
drganghe committed May 10, 2024
1 parent f12e574 commit 6b32c1e
Show file tree
Hide file tree
Showing 23 changed files with 829 additions and 102 deletions.
22 changes: 22 additions & 0 deletions docs/lectures.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -434,6 +434,28 @@ <h5 class="no-anchor card-title listing-title">
</div>
</a>
</div>
<div class="g-col-1" data-index="8" data-listing-date-sort="1715400000000" data-listing-file-modified-sort="1715362217066" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="5" data-listing-word-count-sort="896">
<a href="./lectures/lecture9.html" class="quarto-grid-link">
<div class="quarto-grid-item card h-100 card-left">
<p class="card-img-top">
<img loading="lazy" src="./lectures/images/program-evaluation-ethics-featured.webp" class="thumbnail-image card-img" style="height: 150px;">
</p>
<div class="card-body post-contents">
<h5 class="no-anchor card-title listing-title">
Lecture 9 Ethics, Principles, and Standards
</h5>
<div class="card-attribution card-text-small justify">
<div class="listing-author">
Gang He
</div>
<div class="listing-date">
May 11, 2024
</div>
</div>
</div>
</div>
</a>
</div>
</div>
<div class="listing-no-matching d-none">
No matching items
Expand Down
Binary file added docs/lectures/images/Simplified-Logic-Model.jpeg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/lectures/images/moral-machine-design.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
217 changes: 147 additions & 70 deletions docs/lectures/lecture9.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
div.csl-bib-body { }
div.csl-entry {
clear: both;
margin-bottom: 0em;
}
.hanging-indent div.csl-entry {
margin-left:2em;
Expand Down Expand Up @@ -375,107 +376,183 @@ <h1 class="title">Lecture 9 Ethics, Principles, and Standards</h1>

<p class="date">May 11, 2024</p>
</section>
<section id="recap-lecture-1" class="slide level2 center">
<h2>Recap lecture 1</h2>
<section id="todays-agenda" class="slide level2 center">
<h2>Today’s agenda</h2>
<ul>
<li>Get to know each other</li>
<li>John Snow and evidence based analysis</li>
<li>Evidence based policy
<li>Course evaluation (<a href="http://baruch.cuny.edu/EVALS" class="uri">http://baruch.cuny.edu/EVALS</a>)</li>
<li>Review the class</li>
<li>Ethic considerations</li>
<li>Q&amp;A for presentation and project</li>
</ul>
</section>
<section id="your-evaluation-is-important-to-us" class="slide level2 center">
<h2>Your evaluation is important to us</h2>
<center>
<a href="http://baruch.cuny.edu/EVALS" class="uri">http://baruch.cuny.edu/EVALS</a>
</center>

<img data-src="../lectures/images/Simplified-Logic-Model.jpeg" class="quarto-figure quarto-figure-center r-stretch"><div class="footer">
<p>Source: <a href="https://freshspectrum.com/evaluation-cartoons/">Freshspectrum</a></p>
</div>
</section>
<section id="mindmap-of-class" class="slide level2 center">
<h2>Mindmap of class</h2>
<iframe width="980" height="600" src="../lectures/program-evaluation-markmap.html" title="Program Evaluation Mindmap"></iframe>
</section>
<section id="criticspitfalls-in-evaluations" class="slide level2 center">
<h2>Critics/pitfalls in evaluations</h2>
<ul>
<li>Global clean energy supply chain</li>
<li>Queens curb side organics</li>
</ul></li>
<li>Capturing the wedage</li>
<li>Governers Island and other cases</li>
<li>Data</li>
<li>Sampling</li>
<li>Design/causal models</li>
<li>Implementation</li>
<li>Visulization/Presentation</li>
<li>Generalization</li>
</ul>
</section>
<section id="recap-lecture-2" class="slide level2 center">
<h2>Recap lecture 2</h2>
<section id="recommendations-and-suggestions" class="slide level2 center">
<h2>Recommendations and suggestions</h2>
<ul>
<li>Types and applications of evaluation</li>
<li>Exploratory evaluation</li>
<li>High Line Park</li>
<li>Cultural responsive evaluation</li>
<li>Case: Northwest Housing Alternatives</li>
<li>Evidence-based (findings)</li>
<li>Compliance</li>
<li>Problem solving (causes)</li>
<li>Consider options</li>
<li>Best practices</li>
</ul>
<div class="footer">
<p>Source: <span class="citation" data-cites="newcomer_handbook_2015">Newcomer, Hatry, and Wholey (<a href="#/references" role="doc-biblioref" onclick="">2015</a>)</span></p>
</div>
</section>
<section id="recap-lecture-3" class="slide level2 center">
<h2>Recap lecture 3</h2>
<section id="writing" class="slide level2 center">
<h2>Writing</h2>
<ul>
<li>Theory of change</li>
<li>Logic model</li>
<li>DAG: Directed Acyclic Graphs</li>
<li>Causal models</li>
<li>Case: Jobs Plus in NYC case</li>
<li>Ask a good question game</li>
<li>Report</li>
<li>Abstract (200 words)</li>
<li>Executive summary (2 pages)</li>
<li>Poster/presentation/social media post</li>
</ul>
<div class="footer">
<p>Source: <span class="citation" data-cites="newcomer_handbook_2015">Newcomer, Hatry, and Wholey (<a href="#/references" role="doc-biblioref" onclick="">2015</a>)</span></p>
</div>
</section>
<section id="hippocratic-oath-first-do-no-harm" class="slide level2 center">
<h2>Hippocratic Oath: First do no harm</h2>
<blockquote>
<p>I will use those dietary regimens which will benefit my patients according to my greatest ability and judgment, and I will do no harm or injustice to them. Neither will I administer a poison to anybody when asked to do so, nor will I suggest such a course. Similarly I will not give to a woman a pessary to cause abortion. But I will keep pure and holy both my life and my art. I will not use the knife, not even, verily, on sufferers from stone, but I will give place to such as are craftsmen therein.</p>
</blockquote>
<div class="footer">
<p>Source: <a href="https://en.wikipedia.org/wiki/Hippocratic_Oath">Wikipedia</a></p>
</div>
</section>
<section id="recap-lecture-4" class="slide level2 center">
<h2>Recap lecture 4</h2>
<section class="slide level2 center">

<ul>
<li>Stakeholders</li>
<li>Identify key stakeholder</li>
<li>Engaging stakeholders</li>
<li>Case: ARPR-E</li>
<li>Review of logic models and preliminary discussion of student project ideas</li>
<li>“A fundamental principle is that groups should not be excluded from an intervention that is known to be benefi cial solely for the purpose of conducting an evaluation.”</li>
<li>“if an evaluation shows that a program is cost effective, the funders of the program—whether governments, donors, or nongovernmental organizations—should make reasonable eff orts to expand the program to include the comparison groups once the impact evaluation has been completed.”</li>
<li>“evaluations should not dictate how programs are assigned; instead, evaluations should be fitted to program assignment rules to the extent that those are clear and fair.”</li>
</ul>
<div class="footer">
<p>Source: <span class="citation" data-cites="gertler_impact_2016">Gertler et al. (<a href="#/references" role="doc-biblioref" onclick="">2016</a>)</span></p>
</div>
</section>
<section id="recap-lecture-5" class="slide level2 center">
<h2>Recap lecture 5</h2>
<section id="who-protecting-human-subjects" class="slide level2 center">
<h2>WHO protecting human subjects</h2>
<ul>
<li>Evaluation design</li>
<li>RCT</li>
<li>Quasi-experimental (RD, DiD)</li>
<li>Observational</li>
<li>Proposal</li>
<li>Progress on evaluation project</li>
<li>The rights and welfare of the subjects involved in the impact evaluation should be adequately protected.</li>
<li>The researchers should obtain freely given, informed consent from the participants.</li>
<li>The balance between risk and potential benefi ts involved should be assessed and deemed acceptable by a panel of independent experts.</li>
<li>Any special national requirements should be met.</li>
</ul>
</section>
<section id="recap-lecture-6" class="slide level2 center">
<h2>Recap lecture 6</h2>
<section id="belmont-report" class="slide level2 center">
<h2>Belmont Report</h2>
<ul>
<li>Data sources</li>
<li>Statistical data</li>
<li>Survey</li>
<li>Interview</li>
<li>Evaluation project consultation</li>
<li>Respect for persons. How will the researchers obtain informed consent from their research subjects?</li>
<li>Benefi cence. How will the researchers ensure that the research (1) does not harm and (2) maximizes potential benefi ts and minimizes potential harm?</li>
<li>Justice. How will the researchers ensure that the benefits and burdens of research are fairly and equitably shared?</li>
</ul>
<div class="footer">
<p>Source: <a href="https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/read-the-belmont-report/index.html">The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research</a></p>
</div>
</section>
<section id="recap-lecture-7" class="slide level2 center">
<h2>Recap lecture 7</h2>
<section id="aea-guiding-principles-for-evaluators" class="slide level2 smaller center">
<h2>AEA Guiding Principles For Evaluators</h2>
<ul>
<li>Data analysis</li>
<li>Qualitative methods</li>
<li>Quantitative methods</li>
<li>Visualization</li>
<li>Story telling</li>
<li>Case: NYC G&amp;T Program</li>
<li>Systematic Inquiry: Evaluators conduct data-based inquiries that are thorough, methodical, and contextually relevant.</li>
<li>Competence: Evaluators provide skilled professional services to stakeholders.</li>
<li>Integrity: Evaluators behave with honesty and transparency in order to ensure the integrity of the evaluation.</li>
<li>Respect for People: Evaluators honor the dignity, well-being, and self-worth of individuals and acknowledge the influence of culture within and across groups.</li>
<li>Common Good and Equity: Evaluators strive to contribute to the common good and advancement of an equitable and just society.</li>
</ul>
<div class="footer">
<p>Source: <a href="https://www.eval.org/About/Guiding-Principles">Guiding Principles</a></p>
</div>
</section>
<section id="recap-lecture-8" class="slide level2 center">
<h2>Recap lecture 8</h2>
<section id="conflict-of-interest" class="slide level2 center">
<h2>Conflict of interest</h2>
<ul>
<li>CBA: Cost-benefit analysis</li>
<li>Challenges</li>
<li>Beyond costs</li>
<li>Story: Robert Bullard and Environmental Justice</li>
<li>Case: CBA of EPA Clean Air Act</li>
<li>Family, friendships, financial, or social factors</li>
<li>Impact independent judegement and decision</li>
<li>Disclose and transparent</li>
</ul>
</section>
<section id="todays-agenda" class="slide level2 center">
<h2>Today’s agenda</h2>
<section id="managing-ethical-and-credible-evaluations" class="slide level2 center">
<h2>Managing ethical and credible evaluations</h2>
<ul>
<li>Course evaluation</li>
<li>Review the class</li>
<li>Ethic considerations</li>
<li>Q&amp;A for presentation and project</li>
<li>Protecting the individuals, or human subjects, who participate in the evaluation</li>
<li>Transparency of methods/data</li>
<li>Unbiased, reliable, and credible evaluation</li>
</ul>
<div class="footer">
<p>Source: <span class="citation" data-cites="gertler_impact_2016">Gertler et al. (<a href="#/references" role="doc-biblioref" onclick="">2016</a>)</span></p>
</div>
</section>
<section id="transparent-open-science" class="slide level2 center">
<h2>Transparent: Open science</h2>
<ul>
<li>Publication bias</li>
<li>Data mining</li>
<li>Multiple hypothesis testing</li>
<li><span class="math inline">\(R^2\)</span>/t-test (p-hacking)</li>
<li>Lack of replication</li>
</ul>
</section>
<section id="checklist" class="slide level2 center">
<h2>Checklist</h2>
<div style="font-size:24px">
<ul>
<li>Is assignment to the treatment and comparison groups fair? Are there any groups with particularly high need that should receive the program in any case? Who will be excluded from the impact evaluation?</li>
<li>Has the research team identified the relevant Institutional Review Board(IRB) or National Ethics Review Committee?</li>
<li>Does the impact evaluation schedule allow sufficient time to prepare and submit the research protocol to the IRB and obtain consent before data collection from human subjects begins?</li>
<li>Did the research team submit the research protocol and preanalysis plan to a social science trial registry?</li>
<li>Is a procedure in place to ensure that the key elements of the intervention are documented as they happen, and not only as they are planned?</li>
<li>Do policy makers understand that evaluation results might show that the intervention was not eff ective, and do they agree that such results will be published and not held back?</li>
</ul>
</div>
<div class="footer">
<p>Source: <span class="citation" data-cites="gertler_impact_2016">Gertler et al. (<a href="#/references" role="doc-biblioref" onclick="">2016</a>)</span></p>
</div>
</section>
<section id="story" class="slide level2 center">
<h2>Story</h2>

<img data-src="../lectures/images/moral-machine-design.png" class="quarto-figure quarto-figure-center r-stretch"><center>
<a href="../stories/the-moral-machine-and-ethics.html">The Moral Machine and Ethics</a>
</center>
<div class="footer">
<p>Source: <a href="http://moralmachine.mit.edu" class="uri">http://moralmachine.mit.edu</a></p>
</div>
</section>
<section class="slide level2 smaller scrollable" id="references">

<h3 id="references">References</h3>
<div id="refs" role="list">

<div id="refs" class="references csl-bib-body hanging-indent" data-entry-spacing="0" role="list">
<div id="ref-gertler_impact_2016" class="csl-entry" role="listitem">
Gertler, Paul J., Sebastian Martinez, Patrick Premand, Laura B. Rawlings, and Christel M. J. Vermeersch. 2016. <em>Impact <span>Evaluation</span> in <span>Practice</span>, <span>Second</span> <span>Edition</span></em>. Washington, DC: World Bank. <a href="https://doi.org/10.1596/978-1-4648-0779-4">https://doi.org/10.1596/978-1-4648-0779-4</a>.
</div>
<div id="ref-newcomer_handbook_2015" class="csl-entry" role="listitem">
Newcomer, Kathryn E., Harry P. Hatry, and Joseph S. Wholey. 2015. <em>Handbook of <span>Practical</span> <span>Program</span> <span>Evaluation</span></em>. 4th edition. San Francisco: Jossey-Bass.
</div>
</div>
<div class="footer">
<p>Back to Title <a href="https://drganghe.github.io/program-evaluation/lectures/lecture9/#/title-slide">Slide</a>, All <a href="https://drganghe.github.io/program-evaluation/lectures.html">Lectures</a>, Course <a href="https://drganghe.github.io/program-evaluation/">Home</a>, or Learn More About <a href="https://drganghe.github.io">Gang He</a></p>
Expand Down
Loading

0 comments on commit 6b32c1e

Please sign in to comment.