IMMAP opened less than two years ago, putting Little and his team of faculty, staff and students just a bit ahead of schedule.
The institute, part of the College of Education, runs advanced statistical analyses on research projects from researchers and organizations throughout the country. That is only a small part of its mission, though. IMMAP and its director, who is an internationally known quantitative specialist, are on a mission for social justice through accurate methodology and statistics. To get there, the team is inventing new statistical methods, finding new clients from around the world and training a new generation of specialists to perform the most advanced statistical methods available while breaking ground for more advancement.
鈥淒oing methods right is a matter of social justice,鈥� Little said. 鈥淚f we don鈥檛 do it right, that means the results will be biased and policymakers and practitioners are going to be making decisions based on ill-advised applications of research methods. If we do it right, they鈥檙e going to have the best information.
鈥淭his is a decision science. If you鈥檙e going to be making decisions based on data, you need to make sure the data are presented in their best possible form.鈥�
Starting IMMAPLittle鈥檚 doctorate is in developmental psychology, but most of his career has been spent on the data analysis side of research. He realized in graduate school the importance of producing good, unsullied analytics 鈥� but using only the most commonly accepted statistical techniques didn鈥檛 work for every data set. That approach, however, is what many researchers did then and still do. They learn a few techniques in graduate school and shoehorn their data into one of those techniques, even though it鈥檚 two decades later and technology has advanced to the point the once-imagined techniques are now viable.
Little embraced new techniques, developing some and perfecting others, and over time he made a name for himself as the specialist to call for statistics and methodology questions. For years, his colleagues would invite him to be the methodologist in their grant proposals; he could help design and execute the study so as to reduce statistical errors. The relationship worked for the researcher, who had access to Little鈥檚 expertise throughout the process, and for Little, who earned co-authorship on numerous papers, increasing his stature as an expert in the field.
鈥淭he way grants typically work is they would try to buy a percent of my time,鈥� Little said. 鈥淎fter a certain point I鈥檇 get on two to three grant proposals, but then I would be basically out of percent time they could buy anymore. But there would be three or four other people who would like it if I would participate.鈥�
To date, his name is on more than 250 publications with more than 300 unique co-authors; those publications have garnered nearly 16,000 citations in academic journals.
鈥淏ecause of the quality of the methods, my work is getting out there,鈥� Little said. 鈥淧eople are reading it, people are citing it.鈥�
While a professor at the University of Kansas, administrators asked if he would direct a center for research methods and data analysis. Clients could access his expertise in methodology and innovations in statistical methods, what he called his 鈥�10,000-foot view,鈥� but graduate students and interns could run the actual analyses.
鈥淚 realized if I were to direct such a center, I could be surrounded by people who could do the heavy lifting I don鈥檛 necessarily need to be paid to do,鈥� he said. 鈥淓ffectively, I can participate in many more research projects, provide input and insights that change the science and affect the quality of the work, but do it under the auspices of the institute.鈥�
In 2013 College of Education Dean Scott Ridley contacted Little and invited him to Texas Tech to create IMMAP. He also created an academic concentration (research, evaluation, measurement and statistics, or REMS) to allow students to specialize in quantitative analysis. Many of his clients from Kansas followed him, and he鈥檚 picked up more in the last two years 鈥� enough to add a postdoctorate position and a few more interns.
Katy Roche, an associate professor at the Milken Institute School of Public Health at George Washington University, is one of Little鈥檚 clients. She is researching behavioral and mental health outcomes among Latino children of immigrant parents.
Roche cited Little鈥檚 unmatched experience in statistics and the qualified students and faculty members who are part of his center.
鈥淭odd is uniquely qualified to provide guidance and advice around using state-of-the-art, advanced latent variable modeling techniques; imputing missing data using best practice methods; and developing study designs based on cost-effective approaches to data collection through the use of intentionally missing data,鈥� she said. 鈥淚 would never have the time to gain this knowledge and these skills, so I am most grateful for IMMAP鈥檚 capacity to do this for me.鈥�
IMMAP鈥檚 client list is diverse, including individual professors from Texas Tech and dozens of other universities as well as the U.S. Navy, the Centers for Disease Control and Prevention, the Texas Department of Criminal Justice, the Noyce Foundation and other governmental and nonprofit agencies.
The need for good statisticsMost studies in major academic journals use outdated, less effective data analysis methods, Little said. Considering the reach data may have, and the possible effects as people interpret it, this cannot work.
Knowing this is why Dorothy Espelage, an educational psychologist at the University of Illinois-Urbana/Champaign who is a world-renowned researcher on bullying and power, came to IMMAP. She testified before Congress and other federal committees about bullying. Her data has an effect on national policy, so it needs to be correct 鈥� even more so when she comes up against bullying prevention programs not based on evidence.
She鈥檚 working now on a project that measures the effectiveness of the Second Step program in improving children鈥檚 social-emotional development, which may lead to decreased bullying. The grant is from the CDC and Committee for Children. She attributes her funding in part to Little鈥檚 expertise.
鈥淚t鈥檚 difficult to get funded without a strong statistical team like Todd鈥檚,鈥� Espelage said. 鈥淭hey keep up on the latest methods that I just cannot keep up with and run my large-scale, school-based studies that require a lot of hard work.鈥�
The problem is bigger than researchers using bad methods, however. A couple of years ago a task force for the American Psychological Association found a shortage of quantitative training programs like the REMS program, which means not enough researchers know how to use advanced statistical techniques. Additionally, there are few places to learn new techniques once a researcher finishes graduate school.
To that end, in 2003 Little started Stats Camp, an intensive, weeklong seminar that invites the top quantitative specialists to teach courses on the most advanced statistical techniques available. He routinely brings in students and instructors from throughout the world, and in 2016 there will be Stats Camp Europe for the first time, with Little going to the Netherlands for a week.
He cannot stress enough the importance of good statistics. Little cites social justice every time the question arises: Researchers need to produce valid results, and to do so they need to use the best statistical methods for their question. To do otherwise misleads the public and policy- and decisionmakers.
鈥淭oo often researchers take their question and stuff it into the one or two analytical techniques they know, and it鈥檚 not the right way to do it,鈥� Little said. 鈥淵ou have to tailor the technique to the question.鈥�
IMMAP鈥檚 unique place in academiaThe academic world does not have many IMMAPs or Todd Littles. Some universities have a quantitative specialist who works with other professors on campus, either running their statistical analyses or training researchers on new techniques.
Others have what Little called service centers that can run statistics for a researcher. IMMAP provides this service, but it is a small part of the process.
The relationship between a researcher and IMMAP begins well before the time for data analysis. Little and his team help researchers refine and write the method section of the grant. This helps in two ways: Little has been part of hundreds of successful grants, so he knows what grant review panels look for; and he is the accepted expert on many quantitative techniques. He or one of his team also has invented many of the techniques this type of research uses.
鈥淥ur goal is to make sure the methods section and its execution are described in a pristine manner, such that a grant review panel would have no basis for criticizing the methods,鈥� Little said. 鈥淚n terms of resources, with IMMAP they can鈥檛 ding the resources. And in terms of personnel, we are the experts in the field who have invented most of the methods that we鈥檙e describing, so they can鈥檛 knock the personnel.
鈥淭he only thing they can pick on then is if this is a sufficiently good idea. Would it have the impact we want?鈥�
Invention of new statistical methods also sets IMMAP apart from statistical service centers. Some statistical methods were around 30 years ago but not feasible because the necessary technology didn鈥檛 exist. Others, however, have been developed in concert with technology. That kind of innovation happens almost daily in IMMAP. 鈥淭here are places typically run by statistics departments where you can get help,鈥� Little said. 鈥淭he uniqueness of IMMAP is that we are very involved in the substantive theories being tested; we understand the theory part and that is an attractive feature. Two, we鈥檙e not just applying techniques that have been learned, we鈥檙e innovating techniques. That鈥檚 a one-two punch I think is absolutely unique.鈥� Finally, the experts at IMMAP are more than statisticians. They are quantitative specialists 鈥� well-versed in a statistician鈥檚 work but able to understand the theoretician鈥檚 questions and nature of research. They鈥檙e not looking at the data in a vacuum. In fact, all of the REMS students are required to be proficient in a substantive research area such as developmental or educational psychology.
Missing dataIMMAP and Little are leading the industry in addressing missing data, which all social scientists have in their work. Missing data occurs when, for example, a researcher gets only 400 out of 500 surveys back. Traditionally only the 400 returned surveys are used to analyze the question.
That, however, skews the data set. Those who did respond may feel more strongly about the question than the population as a whole, or the lack of response may represent a specific subset that differs from the population. Either way, the data set does not portray an accurate picture. The missing data must be accounted for.
鈥淚t can wreak havoc if you don鈥檛 treat it properly,鈥� he said. 鈥淲hen you don鈥檛 try to correct for the reason it鈥檚 missing, you have selective data and your results are no longer valid.鈥�
Little鈥檚 work on missing data has presented researchers with ways to address this issue. The methodologies he touts in accounting for biasing effects of missing data is so effective, he said, it will unequivocally increase the validity of the research findings. He has published numerous articles on the subject, and he is working on several projects that continue to advance the utility of modern missing data treatments.
IMMAP staff is developing a software package that facilitates the whole process of accounting for missing data. When it鈥檚 finished it will be open source freeware instead of a copyrighted program, giving all researchers access to it. Again, it鈥檚 a social justice thing for Little.
The REMS degreeCurrently about a dozen graduate students are working toward a REMS degree through IMMAP. Another half a dozen students from throughout the university are employed as graduate research assistants.
Several students are undergraduates who either are earning a minor in quantitative analysis or who want to find out what quantitative analysis is like before they get into graduate school. Little鈥檚 already had one student come to IMMAP when it just opened to check it out and choose to earn the minor concentration, and he thinks others will too, largely because this will practically assure admission into the graduate school of the student鈥檚 choosing.
鈥淢ost academics learn their statistics in graduate school, and then they go out and start their jobs, and if the statistics changed after graduate school, where are they going to learn it?鈥� Little said. 鈥淭here are very few programs like the REMS program where faculty members are dedicated quantitative specialists training students in quantitative methods. As quantitative specialists, their job is to stay on top of and invent new methods.鈥�
RSS