In which an incident is deconstructed and the impressions received and left behind might be more important than what may or may not have actually happened.
So let me be clear from the start: No laws were broken. As far as I can tell, every single requirement and guideline for the ethical use of human subjects was met, at the very least on paper and maybe even, more or less, in person. The sense of unease stems not from deliberate actions (probably), but more due to a potential neglect of questionable benign-ness, or perhaps neatly convenient acts of omission.
It’s just that contextually, based on perhaps singular perceptions and a long history of interest in personal privacy in the digital age, it all seems kind of… sneaky.
And while what follows isn’t really about learning analytics, I can’t help but think that it could be, someday. Full disclosure and voluntary participation are all well and good on paper. But do real-life contexts and circumstances really encourage or allow people to understand the nature and use of the personal data they are casting off as they travel through life… or through educational institutions?
Tensions on the fringe
One can dodge and weave and innovate in learning and education all one wants, but as no innovation or innovator is an island, there regularly appear those moments and phases when one bows, however temporarily, to convention. This includes, among other things, interactions with the Behemoths of Higher Education.
As our household is currently negotiating compromise phases in which, due mostly to issues of timing, some things which may be useful are still locked behind the gates and in the ivory towers, there are, in the concession to convention, ample opportunities to roll over and play dead on issues which otherwise and previously have elicited independent thought. The single, not-dead-yet issue to be addressed here, lover as I am of the revelation of small things, is a small portion of the small segment of higher education adventures known as first-year-student orientation.
Having been a witness/eavesdropper/story listener for two or three and half such events (lack of convention accounts for the odd numbering) over the past year or two (ditto), it is quite clear that these orientations are largely a) tiresomely mandatory and mass produced and b) much more for the convenience of the institution than the student. Everybody in one place (physically), given (verbally and in reams of print, whether it exists digitally elsewhere or not) the same information (relevant to the individual student or not) at the same time.
Quite a shock to more independent-minded learners, but one must, I have intoned parentally, persevere for the potentially greater good. So to the idea of being lead en masse through a university library tour (ignoring the detailed map at the entrance door) and pointed to such fascinations as “computers that contain databases,” I have said: March on. To the apparently ubiquitous indoctrinal lectures on Why Higher Education is Morally Superior to Other Forms of Existence (OK, I may have renamed that a bit), I have said: Mental and written critiques are appropriate (moral) responses. To the “Maintaining a Four Year Graduation Plan” with matrixed requirement charts, I have rolled my eyes.
And to the late-afternoon text from the depths of one such orientation that said “They’re asking me about my religion and your income and if I have ever consumed alcohol or believe in abortion. I’m pretty sure they can’t REALLY ask that… can they?” …. I said “Walruses Tell Fables!?” (or something similar).
Yes, They can: The Freshman Survey
As it turns out, They indeed can ask these things. They do ask these things. And, over the course of the next 15 minutes on the side of the bike trail with mobile device in hand and the guidance of the mobile device and descriptive narrative at the other end of the texts, we discovered that They’ve been doing so since 1966.
Somehow, the Cooperative Institutional Research Program Freshman Survey had escaped my notice. I don’t remember taking it (which may be significant in and of itself), and the other Young College Person in the household has for some reason never seen one. But there it is. A massive trove of highly specific and personal data, gathered from eight and a half million students over 40-plus years. Researched and cited six hundred ways from Sunday over the years (and processed at the current rate of a baseline fee of $775 per institution plus $3 per survey by the Higher Education Research Institute ). This is the Godzilla of youth surveys, a socio-demographic rainbows-and-ponies dream source, with 200,000 students surveyed in 2011 alone.
Even if one were inclined to tilt at windmills and dragons, to criticize this survey and its behemoth backers or its makers or responders or interpreters or yes, its beneficiaries (because one must stand in awe of the sheer massiveness of the data) would be the height of wasted effort. And to clarify: In the big-picture-well-after-the-fact of the actual “survey incident,” let me be clear that the Higher Education Research Institute’s website pertaining to this survey exhaustively addresses the processes and procedures and ethics statements in ways that fulfill all of the necessary categories of due diligence and non-coercion and sincere expressions of desire to gain valid data.
In other words, they have — in print and in public— clearly covered their hindquarters on all of the concerns that arose in that 15-minute window on an early June afternoon. They are untouchable in the world of academic and scientific freedom and responsibility. And no, students do not HAVE to answer the questions. All, some, or none—totally voluntary. Except…
It’s the context beyond the content
There remains this fascinating and disturbing gap, or chasm, or Grand Canyon of Disconnect, between what happens with the ideally ethical survey plan with its detailed CYA statements on a website, and the experience of the (or this particular) surveyed student on the other end of the texts with a folder containing a four-page bubble test and a single page of explanatory fine print. So context becomes critical. How to pick this apart…?
Well, let’s start with the setting and lead-up to the survey administration. At the University of Massiveness, students are corralled in some-hundred batches for 30-some hours and trotted through icebreakers, tours, course advisories, registration processes, and sessions in financial and personal responsibility, etc. Lather, rinse, repeat throughout the summer, guided over multiple weeks by Ernest Older Students/camp counselors.
If I recall the half-hour I spent as a parent in the Welcome speech correctly (and again, impression is greater than reality here), the idea of the survey was introduced or at least referred to in passing by an administrator at the very beginning of the day. How Official. And the Freshman Survey is listed on the day’s events, printed clearly and capitalized (how official) in the multi-colored, high-quality (official) orientation booklet, to take place in a rotation with Dinner and Dorm check-in, depending one’s subgroup. And so by the time the actual surveying occurs, new students have been receiving a day-long stream of authority-based messages about expectations for university students. And, as my texter noted, having been “talked at” for several hours, patience and attentiveness and the ability to absorb any more information was wearing very thin. So here’s where things get tricky. The HERI website clearly states:
If your campus administers the survey in a proctored on-campus setting, the following text should be read aloud to students prior to distributing the instruments. While you are free to personalize or edit this text, the points in bold must be included:
“We ask that you complete this survey as part of a national study of college students conducted by the Higher Education Research Institute at UCLA. One major goal of this research is to determine what happens to students when they attend college. This study is designed to help improve the quality of the college experience and thus may benefit future generations of students. Results of your participation also will be directly beneficial to students at [name of your campus], since we will receive complete tabulations of your responses to compare with the responses of students at other institutions. We ask for your name and address so that the researchers at UCLA can contact you at some later date for a follow-up study. We would also like your Student ID Number so we can merge your responses with other campus data to support our institutional research program. You also may complete this survey on the Internet—please refer to the attached instructions if you wish to do so. Your responses will be used only for research purposes and will be strictly confidential. Please read the ‘CIRP Freshman Survey Information Sheet’ for more information about your rights as a participant in this research. Your participation is voluntary and will not affect your standing at [name of your campus].”
And here’s what I know from the texts I received and the subsequent conversation upon arrival home: the texter has absolutely NO recall that that this statement was read. True, we can’t say that it was or it wasn’t. But after six or seven hours of information and the promise of dinner just around the corner, how many of the students were cognitively equipped to hear and understand the message and its implications even if it was read?
And yes, you read that right:
“We ask for your name and address … [and] your Student ID Number…” (CYA website disclosure says: “Before receiving any responses your school is required to certify in advance that the data will only be used for research purposes will not be used to investigate specific individuals.” Um, so California can call you personally, but your local school promises not to? And what about that loaded word…”investigate”?)
Familiarity and disorientation
And what do we know about these named and numbered (but we’ll hide our eyes, really we will) first-year students, survey or not? Well, I’d surmise that most got there by playing the Educational Accomplishment game of tests and coursework well enough to be accepted as one of the eight applicants for every seat—that was an important point made during the Welcome speech. We also know that these students are the direct legacy of the No Child Left Behind era, where testing has been of huge importance, from yearly progress measurements to the absolutism of the AP testing sold as the pinnacle of success.
Most of them have not been away from home for very long (many, based on a show of hands, were looking at life in the “big city” for the first time and over half of them had never done their own laundry), and many of the parents were just across campus on their own “orientation” track, where they were, based on the schedule, being reassured about the university’s ability to act in loco parentis. One could also surmise that most students were about 18 years old and a week or two out of high school.
The reason I detail this is that while I have a strong belief in and first-hand knowledge of the potential of young people to think critically and innovatively when sincerely offered the necessary freedoms and opportunities to do so, I can’t help but think that none of the characteristics noted here really set up these young students to critically reflect on what they were being asked to do. After a steady cultural diet of “college is the ultimate goal,” with the “college selection process” fraught with extreme family and school anxiety and then focusing on new-found loyalty to mascots and colors and the commitment of great sums of money… well, certainly it is preferable to believe that anything your college feeds you is delicious and nutritious. Including the survey in its official time slot.
And let’s talk about the survey administration method “preferred” by HERI:
“The best results occur when the survey is administered in a proctored setting.”
Which means, like any “test,” in a formal setting. In a classroom. With someone “in charge” watching you. And other students lined up next to you in their desks. The texter later noted she “felt like” They said something about students not being able to leave until they had completed the survey, but then realized They could just as well have said, “You can leave when you are done,” in which “done” (in other words, a completed survey) could be inferred as a necessary requirement for departure.
And the survey itself? In this case, a fill-in-the-bubble multiple choice, just like every other Very Important and Official test given to students to measure their yearly progress beginning in kindergarten or to assess their college readiness. I think one can’t underestimate the Pavlovian response “high-achieving” students such as those accepted by the University of Massiveness might have to another multiple choice test.
It must feel very familiar. And mandatory. Which might explain an astonishing 90 percent-plus national completion rate, if I understand correctly.
And finally, there was the content of the survey itself… which is where the eyebrows began to rise in the first place.
Oversharing… or oversensitivity?
Here are some questions from the survey… (and some cantankerous interpretations):
What is your best estimate of your parents’ total income last year? (Report on your parents’ personal information without their knowledge.)
For the activities listed below, indicate which ones you did during the past year [by frequency]?
…“drank beer/”wine or liquor” (Would you like to admit to doing something illegal?)
… “was a guest in a teacher’s home” (Just… creepy)
…”Failed to complete homework on time/“came late to class”/ “fell asleep during class” (How compliant are you and can we really expect much of you?)
Also included: political litmus ratings scales for issues such as:
Abortion
Death penalty
Racial discrimination
Taxing the wealthy
Same-sex marriage
Free speech
And questions relating to mental health, with references to depression, emotional health, self-understanding, and counseling.
Not to mention questions that initially seem truly a shot in the dark: personal opinions about future plans regarding degrees, life goals, philosophical views, the nature of success, etc. Why ask such nebulous, non-verifiable things? Because wait, there’s more. HERI has THREE MORE SURVEYS (not that those new, proctored students know this) that can be administered during a student’s college career. And all of a sudden the massiveness and cross-validation of the data on both the cohort and each individual takes on proportions the mind simply fails to grasp.
True, it might not be “worth it” to care all that much. Between Facebook and financial aid forms, a lot of the survey info could be obtained otherwise, although with much less ease.
Are the survey responses interesting and valuable as a picture of society in evolution, among other things? Yes, yes of course. Is there anything necessarily bad about sharing this kind of info? Maybe not. But is it really something people want to put out there with the assumption that it will forever and ever be treated benevolently by the universe? Would this data make great political fodder when the database is hacked or leaked and the responses from individual students are plumbed? Oh. My. And if cooperative students fill out surveys like this at regular intervals, are they fully equipped to grasp the implications of this first step going forward when they are at the end of a long day of potentially disorienting orientation, even if the “bigger picture” were highlighted for them?
Keeping the uncooperative ten percent at ten
It is the Great Trade-Off. As a researcher or an institution, data is gold, and everyone can sleep better by not scrutinizing too closely any issues of context if you’ve otherwise met all of the research ethics guidelines. It’s just easiest to deliver this survey, sanctioned by all sorts of Powers that Be over time as it is, without highlighting any of this messiness, isn’t it? The extraction of personal data is made incidental and relegated to background noise—just another “thing” in the course of the day. And as the survey subject, well, just close your eyes, and it will soon be all over, and unless you think about it (or the Queen, perhaps), you’ll never know or recognize exactly what you’ve given away or revealed. Unless… well, the “unless” is up to your imagination.
And ultimately, why should I, personally, make a fuss of several hundred words? After all, my afternoon texter, who has an apparently unusually acute sense of personal privacy, refused to be seduced into complacency by the Dark Lords of Data (or, optimistically, the Well-Meaning Minions of Measurement). But in this stance, she also had:
1.) The reflective/critical capacity to consider the implications of what was being asked and wonder why.
2.) The curiosity to seek further information by locating and plowing through the “Survey Information Sheet” (until then ignored in the mass of other paperwork) that revealed the optional-ness of participation.
3.) The ability to use a mobile device to obtain information and consultation outside the immediate context to further validate her questioning and understanding, and
4.) The self-assuredness to walk out of a full room of peers under authoritative supervision without handing in her survey. (At which time her earnest seat neighbor noted she was texting her mother to find out which family income category she should mark.)
I would think higher education researchers in particular would be more interested in understanding the development of these skills and attitudes than in homework and partying habits. But chances are, this particular subject probably would react the same way to a survey about that as well.
One other small point: This intended survey subject/texter happened to be several solstices short of legal age. We generally note that if you’re invited to and intend to hold your own in unorthodox ways, you do it without playing the age card. Age is, however, an issue which is addressed on the HERI website (nothing if not thorough), where it is noted that parental permission may be necessary for minors to participate in the survey. But again, this was not addressed in the immediate context of taking the survey at the University of Massiveness. So the written information (for those who chose to read it) that states “Your participation and return of the questionnaire indicate your consent to participate in the study” seems a bit disingenuous. Especially since there are bubbles to mark on the survey which acknowledge under-18 students. And what about those tired, 18-plus but perhaps cognitively overloaded students who have just been thoroughly oriented/socialized into compliance with university norms? It seems there is no room for second thoughts once that survey leaves their hands…
Thinking Out Loud 
Leave a Reply