In 2004, the National Center for Engineering and Technology Education (NCETE) secured funding from the National Science Foundation (NSF) to infuse engineering design into the schools through technology education. In order to reach this goal the researchers, in cooperation with NCETE, conducted a two phase study to identify outcomes for high-school students studying engineering. The first study (referred to as a Phase I) focused on students who intended to enter an engineering program after high school, answering the question:
What are the engineering student outcomes that prospective engineering students in grades 9 through 12 should know and be able to do and prior to entering into a post-secondary engineering program?
This initial study by Childress & Rhodes (2008) started with preexisting items selected from ten sources, including focus groups and national standards projects. At the end of the Delphi Round 3, very few of these items had been dropped since the consensus on all of them was high. Therefore, the researchers decided to have selected engineers categorize the outcome items into groups of conceptual likeness and to assign categorical names to the groupings. These groups then formed the basis of the instruments for the remaining Delphi rounds.
This resultant baseline of achievement outcomes for prospective engineering students was then used to design a modified Delphi instrument for the second study (Phase II) reported herein. This study focused on students who were enrolled in technology education for general education purposes and sought input from technology education teachers, teacher educators, and supervisors regarding the following question:
What are those engineering outcomes that should be taught in a high school technology education program in which the focus is general technology education program in which the focus is a general technological literacy and not pre-engineering?
Wicklein’s “Five good reasons for engineering design as the focus for technology education” (2006; cf. NAE, 2004; Lewis, 2004) effectively identified the importance of this study. In presenting the advantages of infusing engineering into technology education, he stated that the optimization, analysis, and predictions are among the more important things and those students should know about engineering. This study takes the next step by identifying the related engineering design concepts to be included in a technology education curriculum.
Among recent efforts to identify engineering outcomes for high school students. Dearing and Daugherty (2004) used a modified Delphi technique to identify concepts that would prepare secondary students for postsecondary engineering, in the context of a course promoting technological literacy. Their experts were secondary and postsecondary educators.
To develop the Delphi instrument for the present study, the results of Phase I were categorized and a ranked list of 40 outcomes, recommended by engineers for high school students who want to pursue postsecondary engineering education, was determined.
Twenty two technology educators served as Phase II Delphi participants: 7 high-school teachers, 6 administrators and 9 teacher educators (Table 1). Each was paid a participation stipend. Participants were selected based on the researchers' knowledge of their professional expertise, or in some cases, on the recommendation of a person deemed qualified for participation. Potential participants were asked to describe their level of knowledge of engineering and its relationship to technology education (Table 2).
|Female||13.6%||( n = 3)|
|Male||86.4%||( n = 19)|
|Caucasian||95.5%||( n = 21)|
|African American||0%||( n = 0)|
|Hispanic||4.5%||( n = 1)|
|Range||26 to 62|
|Years Teaching Experience|
|Range||4 to 37|
|Years Experience in Current TE Role|
|Range||4 to 31|
|ID No.||Yrs. Experience||Position||Self-Description of Knowledge|
|1||7||Classroom teacher||Described STLs and relationship to engineering|
|3||7||Classroom teacher||Past president of TE related association|
|4||7||Classroom teacher||BS in Mechanical Engineering|
|5||4||Classroom teacher||MS in Bioengineering|
|7||36||Supervisor||Taught engineering for 15 years; curriculum writer|
|8||32||Supervisor||Chair of a standards team; wrote cross reference of standards with PLTW|
|9||6||Classroom teacher||Did specific research on it in college; writing standards for state|
|10||19.5||Classroom teacher||Role is to prepare students to major in engineering|
|11||36||Teacher educator||Consulted with engineering firms regularly, co-inventor of medical research apparatus, keep in close contact with engineering departments|
|12||37||Teacher educator||Have studied and written about engineering concepts|
|13||25||Teacher educator||Unsure. Called for follow-up. Is not an engineer but understands role of engineering in technology education|
|14||29||Teacher educator||Above average|
|15||18||Classroom teacher||Teach engineering|
|16||31||Teacher educator||Know enough to be dangerous Works with College of Engineering and has published and consulted on engineering related projects.|
|17||15||Teacher educator||Understand principles of engineering through applied teaching and learning activities and involvement|
|19||27||Supervisor/Classroom teacher||Understands the basic engineering process and tools needed to be an engineer like math and teamwork skills, knowledge of specific engineering fields is very minimal|
|20||30||Teacher educator||On a scale of 1 to 5, I am a 4 – better than average, not expert|
|21||33||Teacher educator||Very high|
No one participating in the study was an employee of an institution that is a full partner in the NCETE. Participant #4 and #22 dropped out at Round 1. All participants who completed Round 3 were paid a fee for their full participation.
In Rounds 1 through 3, participants were given the list of outcomes items from Phase I. They were informed that the items had been selected by engineers for inclusion in high school pre-engineering programs. The Phase II participants were asked to rate these items, reword them if needed, add and rate new items, and provide comments. They were also asked to rank the importance of the items using a five point scale (Least Important, Less Important, Important, More Important, Most Important; Clark & Wenig, 1999). The descriptions for each of these categories are reported in Table 3. The interquartile range (IQR) was used as the statistic for variability of rating responses; an IQR of 1 was determined by the researchers to indicate consensus on an item (Wicklein, 1993).
|1.||Least Important: Not necessary for inclusion in a technological literacy program.|
|2.||Less Important: Less than necessary for inclusion in a technological literacy program.|
|3.||Important: Necessary for inclusion in a technological literacy program.|
|4.||More Important: Essential for inclusion in a technological literacy program.|
|5.||Most Important: Most essential for inclusion in a technological literacy program.|
Thirty-five of the 44 items from Phase I achieved consensus after Round 3. Three additional items were identified by participants after Round 1 and these items achieved consensus after Round 2. After Round 3, consensus had been reached on 38 engineering outcome items. Eleven items were rated as “Important” to include in a technological literacy program, 22 items were rated “More Important” and five items were rated “Most Important.” For comparison, after three rounds the participants in Phase I reached consensus on what should be taught 1st, 3rd, and 7th places in order of importance. However, in Phase II after three rounds, the participants reached consensus on the order of only one grouping, Emerging Fields of Engineering, and it was ranked lowest in importance. In Phase II, no engineering outcomes items were dropped due to a low median importance ratings; however, those outcome items that were not included in the final list were those for which consensus was not reached. The final engineering outcome item ratings and group rankings for Phase II are presented in Table 4. A complete statistical analysis of all data, (including non-consensus items) is available at http://www.ncete.org/flash/Outcomes.pdf.
Group: Engineering Design
Regarding engineering outcomes for Engineering Design the student in grades 9-12:
|* Indicates a consensus item added by the participants themselves.|
|4||Rank Undetermined||Is aware of how engineering principles must be applied when designing engineering solutions to problems.|
|4||Understands that creativity is an important characteristic for engineers to apply in design.|
|4||Believes in his/her ability to design a solution to a problem.|
|5||Recognizes that there are many approaches to design and not just one “design process.”|
|3||Understands engineering as it is actually practiced as a future career option.|
Group: Application of Engineering Design
Regarding engineering outcomes for Application of Engineering. Design the student in grades 9-12:
|4||Rank Undetermined||Designs, produces, and tests prototypes of products.|
|4||Understands that there is no perfect design. Designs that are best in one respect may be inferior in other ways (cost or appearance). Usually some features must be sacrificed as trade-offs to gain other features.|
|3.5||Conducts reverse engineering and can analyze how a product or process was designed and created.|
|4*||Understands how to work well on multidisciplinary teams.|
|4||Applies research and development and experimentation in the production of new or improved products, processes, and materials.|
Group: Engineering Analysis
Regarding engineering outcomes for Engineering Analysis the student in grades 9-12:
|4||Rank Undetermined||Uses models to study processes that cannot be studied directly.|
|4||Applies mathematics and science to the engineering process.|
|4||Understands that knowledge of science and mathematics is critical to engineering.|
|3||Uses a physical or mathematical model to estimate the probability of events.|
|3||Uses optimization techniques to determine optimum solutions to problems.|
Group: Engineering and Human Values
Regarding engineering outcomes for Engin. and Human Values the student in grades 9-12:
|4||Rank Undetermined||Practices engineering ethics.|
|5||Is aware of how societal interests, economics, ergonomics, and environmental considerations influence a solution.|
|5||Understands how other factors, such as cost, safety, appearance, environmental impact, and what will happen if the solution fails must be considered when designing engineering solutions to problems.|
|4||Takes human values and limitations into account when designing and solving problems.|
|5||Understands that the solution to one problem may create other problems.|
Group: Engineering Communication
Regarding engineering outcomes for Engin. Communication the student in grades 9-12:
|4||Rank Undetermined||Understands basic personal computer operations and uses basic computer applications such as word processors, spreadsheets, and presentation software.|
|4||Provides basic technical presentations, graphics, and reports, and communicates verbally information related to engineering processes|
|4||Uses technical drawings to construct or implement an object, structure, or process.|
|4||Visualizes in three dimensions.|
|3||Understands computer-aided engineering.|
|3||Applies the rules of dimensioning and tolerancing.|
|4||Uses computer-aided design to construct technical drawings.|
Group: Engineering Science
Regarding engineering outcomes for Engineering Science the student in grades 9- 12:
|5||Rank Undetermined||Develops basic ability to use, manage, and assess technology.|
|3||Applies knowledge of basic ergonomics to the engineering process.|
|4||Develops basic skill in the use of tools for material processes.|
|4||Applies basic power and energy concepts.|
|3.5||Applies knowledge of the processes for manufacturing products to the engineering process.|
|4||Applies knowledge of material processes to the engineering process.|
|4||Applies knowledge of basic mechanics to the engineering process.|
|3||Applies knowledge of basic dynamics and motion of rigid bodies and particles to the engineering process.|
|4*||Understands open and closed loop systems.|
|3*||Describes the sources, basic chemical structure, recycling potential, and environment al impacts of widely used industrial materials.|
Group: Emerging Fields of Engineering
Regarding engineering outcomes for Emerging Fields of Engin. the student in grades 9-12:
|3||Rank 7 th||Understands the importance of nanotechnologies in developing the next generation of innovations (less power, smaller).|
As indicated in Table 5, the consensus rate for technology educators in Phase II was very similar to the consensus rate for the engineers in Phase I.
|Round||Tech Educators (Phase II)||Engineers (Phase I)|
With the exception of the Emerging Fields of Engineering group, it was difficult for both groups of participants (engineers and technology educators) to reach consensus on outcome grouping. Among the plausible reasons are that both phases dedicated only three rounds to consensus building for the groupings, and the initial groupings were juried instead of being crafted by a complete Delphi process.
As of Round 3, there are several consensus items that provide reinforcement of the importance of the engineering design processes that the NCETE has selected as its professional development focuses: constraints, optimization, prediction, and analysis (COPA). Other technology educators may be interested in these outcomes also. Those items are presented in Table 6.
|Round 3||Round 2||Round 1|
|14*||Uses models to study processes that cannot be studied directly.||0*||4||.49||.75*||4||.92||1*||4||1.05|
|15*||Uses optimization techniques to determine optimum solutions to problems.||1*||3||.51||1*||3||.74||1*||3||.98|
|16*||Applies mathematics and science to the engineering process.||.75*||4||1.02||1.75||4||1.09||1.25||4||1.01|
|17*||Uses a physical or mathematical model to estimate the probability of events.||1*||3||.85||1.75||3||1.05||1.25||3||1.09|
|19*||Understands that knowledge of science and mathematics is critical to engineering.||1*||4||.95||1*||4||.95||1*||4||1.00|
The use of models for indirect study such as analysis or prediction (Item 14) had an IQR of zero. Typical comments in support of the More Important rating include the following:
Optimization (Item 15) was rated as Important to include in a technological literacy program. Comments related to optimization include the following:
Mathematical analysis-related Items 16 and 19 and predication-related Item 17 were rated More Important and Important respectively to include in a technological literacy program. Supporting comments related to the use of mathematics and science for the analysis items included the following:
Comments supporting the prediction outcome included the following:
|Round 3||Round 2||Round 1|
|21*||Is aware of how societal interests, economics,…environment al…influence a solution.||1*||5||.604||1*||5||.606||1*||5||.790|
|22*||Understands how other factors, such as cost…must be considered when designing…||1*||5||.995||1*||5||1.08||1*||5||1.07|
Constraints (Items 21 and 22) were rated as Most Important to include in a technological literacy program. See Table 7. Comments in support of the importance of constraints were.
Technology educators, interestingly, rated approaches to the design process differently than did engineers. The technology educators also rated technical drawing slightly higher than did the engineers. After Round 3 for this grouping, the engineers ended with an IQR of 1 while the technology educators has an IOR of zero, as required in Table 8.
|Round 3||Round 2||Round 1|
|6*||Recognizes that there are many approaches to design and not just one “design process.”||1*||5||.605||1*||5||.605||1*||5||1.06|
|28*||Uses technical drawings to construct or implement an object, structure, or process.||0*||4||.459||0*||4||.887||.5*||4||.669|
Supporting comments related to recognizing that there is a variety of design processes included:
Comments related to support of the technical drawing outcome included.
These findings were similar to those of Dearing and Daugherty in their modified Delphi study.
Technology educators added an item that did not appear on their Round 1 instrument because it was not a consensus item for the engineers. However, it did appear on the Round 1 instrument for the engineers. This item relates to the NAE’s prediction that working on teams will be very important to engineering in the future (Item 47 below in Table 9).
|Round 3||Round 2|
|Added 47* add to Grp 2||Understands how to work well on multidisciplinary teams.||1*||4||.67||1*||4||1.07|
Comments related to support of the multidisciplinary teams outcome included:
Technology education participants did not reach consensus on the importance of managing the engineering design process. Related comments are included following Table 10 below.
|Round 3||Round 2||Round 1|
|8||Organizes and manages the engineering. design process that includes optimal use…materials. processes…||2||4||1.02||2||3.5||1.03||2||4||1.12|
Dearing and Daugherty (2004) described a modified Delphi study that they conducted with technology teachers, technology teacher educators, and engineering educators. The purpose of the study was to identify those curricular concepts that are necessary to teach high school students in order to prepare them for postsecondary engineering education, while preserving the mission of teaching technological literacy. Dearing and Daugherty developed a predetermined list based on information from Project Lead the Way, CORD’s Principles of Technology, The Standards for Technological Literacy, ASEE, and others. Participants were to decide if a concept should be included in such a curriculum.
Dearing and Daugherty measured consensus in terms of an item’s standard deviation. Fifty-two concepts on their list met the criterion for consensus and were retained. While the purposes of the Dearing and Daugherty study differed from this study, they were both focused on technological literacy. Twenty-nine of the engineering consensus items in the Dearing and Daugherty study overlap conceptually with the consensus items in this study.
Finally, it should be noted that the Corporate Member Council of the American Society for Engineering Education has been working on a set of “…National Content Standards for K-12 Engineering/Engineering Technology…” ( Morrison, 2007 , p. 1). Most of the 70 outcomes derived from these standards overlap with the findings of this study. However, it should be noted that the purposes of the Corporate Member Council’s study differ from the purpose of this study. The final findings of the Corporate Member Council’s study were not available at the time this article was written.
The following recommendations are offered to technology education teacher educators, technology education teachers, and technology education administrators.
There are at least two primary points of view regarding the application of the items identified in this study to general technological literacy programs—those that are not pre-engineering. One is that a curriculum could be designed that is heavily influenced by engineering contexts. Those outcome items identified herein that overlap with the Standards for Technological Literacy would address the same standard, but the delivery would be set in the context of engineering. A second point of view is that the outcomes identified herein would be used sparingly in a program intended to develop technological literacy; engineering would be one of many contexts and topics of study included. Although agreement was reached on only a few groupings, they may still be of use to teachers insofar as the engineers agreed that Group 1, Engineering Design, was the group of outcome items of primary importance in a curriculum that is crowded and has limited time to dedicate to engineering outcomes. Likewise, Group 7, Emerging Fields of Engineering, represents the least important grouping that one would want to teach if time were limited. Finally, a crowded curriculum will actually provide less opportunity for students to learn about engineering in meaningful ways—meaningful ways that will tend to attract underrepresented populations to STEM areas. Just because consensus was reached on as many as 38 different outcome items in this study does not mean that all of them should be taught in one course. They should be applied as needed and when pertinent over the span of the ninth through twelfth grades.
Craig Rhodes (email@example.com) is an Associate Professor and Vincent Childress (firstname.lastname@example.org) is a Professor in the Technology Education Program at North Carolina A&T State University, Greensboro
Childress, V. W., & Rhodes, C. (2008). Engineering Outcomes for Grades 9-12. The Technology Teacher. 67 (7), 5-12.
Clark, A. C., & Wenig, R. E. (1999). Identification of quality characteristics for technology education programs: A North Carolina case study. Journal of Technology Education, 11 (1). Retrieved September 23, 2005 from http://scholar.lib.vt.edu/ejournals/JTE/v11n1/clark.html
Dearing, B. M., & Daugherty, M. K. (2004). Delivering engineering content in technology education. The Technology Teacher, 64 (3), 8-11.
Lewis, T. (2004). A turn to engineering: The continuing struggle of technology education for legitimization as a school subject. Journal of Technology Education, 16 (1). Retrieved June 25, 2005 from: http://scholar.lib.vt.edu/ejournals/JTE/v16n1/lewis.html
Morrison, R. E. (2007). Personal communication.
National Academy of Engineering (2004). The engineer of 2020: Visions of engineering in the new century. Washington, DC: National Academies Press.
Wicklein, R. C. (1993). Identifying critical issues and problems in technology education using a modified-Delphi technique. Journal of Technology Education, 5 (1). Retrieved September 23, 2005 from http://scholar.lib.vt.edu/ejournals/JTE/v5n1/wicklein.jte-v5n1.html
Wicklein, R. C. (2006). Five good reasons for engineering design as the focus for technology education. The Technology Teacher, 65 (7), 25-29.
The researchers would like to thank the participants in this study for their dedication and thoughtfulness. The researchers would also like to thank the National Center for Engineering and Technology Education for its support of this study.