Navigation – Plan du site

AccueilNumérosIV - 1Symposia. Pragmatism and the Soci...Section III. Contemporary Appropr...Critical Pragmatist and the Recon...

Symposia. Pragmatism and the Social Sciences: A Century of Influences and Interactions, vol.2
Section III. Contemporary Appropriations

Critical Pragmatist and the Reconnection of Science and Values in Educational Research

Walter Feinberg

Résumé

Randomized field experiments, which in the United States has been proposed as the gold standard of educational research, (National Research Council, 2002) is dismissed by some critics as “positivistic.” Although this dismissal over identifies positivism with a specific research method, the larger point is accurate: the “gold standard” is often insensitive to local situations and human value and philosophical positivism supports and encourages this insensitivity. In this paper I examine the way positivism is limited in terms of its understanding of the role of values in educational research and I offer pragmatism as a productive alternative to these limitations. In contrast to the view of some critics I show that pragmatism would not reject out of hand randomized field experiments. Rather it would contextualize them as one of a number of valuable research tools. I argue here that pragmatism provides a more complete understanding of the research process because unlike positivism it does not dismiss value claims as meaningless, but provides a way to rationally address them. Thus its understanding of the research process is better suited to the process of educating which is inherently infused with values. I also expand on the ideas of traditional pragmatism by introducing a variation that I call critical pragmatism.

Haut de page

Notes de l’auteur

My appreciation to the Spencer Foundation for supporting this project, to Fudan University for inviting me to lecture on this topic, to Denis Phillips for his comments on positivism and to Eric Bredo for his very comprehensive and insightful critique of an earlier draft.

Texte intégral

I. Meaning and Nonsense: A Review of the Basic Tenets of Positivism

1A brief review of the familiar idea of positivism and post positivist revisions will help to ground my argument. For the positivist there are but two types of meaningful statements. Those called “analytic” are true (or false) by virtue of conformity (or non-conformity) to a definition. “One Plus one equals two” and “all bachelors are unmarried men” are common examples of analytic statements. Those called synthetic or empirical statements are those that are true (or false) by virtue of conformity (or non conformity) to experience. “There is one bachelor in this room” and “The Chinese population is over a billion people” are both synthetic statements. For the positivist therefore scientific statements are a form of empirical claim and as such they need to be verified, in principle, by some directly observable or some inferred event that is directly observable. There are differences within positivism about the status of the object of knowledge and whether scientific claims actually mirror reality, or simply record information and try to describe patterns. By allowing only empirical and analytic statements to be meaningful it discounts value claims relegating them to what it would call mere preferences – like whether you prefer chocolate to vanilla – or to ejaculatory sounds like a sigh or a chuckle. They are like the kind of noise that people make when they are perhaps overly stimulated. In a cleaver play on words these are labeled “non sense” – signifying both meaningless gibberish and assertions that have no sense content.

II. Post Positivism

2Many post-positivist agree with the critics of classical positivism that the claims that propositions are meaningful only if they can be subject to verification through experience, is too strict and too neat. Too neat because as Imre Lakatos (Lakatos 1970) has shown, scientists will often protect their pet but immature and potentially productive theories like a hawk protects her immature chicks, using non-rational as well as rational weapons to ward off real threats. For Lakatos the emotional element has an important function in enabling new theories to develop while they are young and vulnerable to criticism by older, more established ones. Traditional positivism is also too strict because as Quine (Quine 1953) has nicely shown, any theoretical claim, even those of logic and math, is always open to challenge. For Quine this also holds even for direct observation, like pointing (Quine 1960: 1-26).

  • 1 While this may seem less true of Lakatos who allows that personal ambition may motivate the scienti (...)

3The recent history of philosophy of science has been one of relaxing the standard of what to count as science, first from the idea of verification advanced by A. J. Ayer (Ayer 1936), to the notion of falsification provided by Popper (Popper 1959), to the notion of the progressive nature of an overall research program as developed by Lakatos (Lakatos 1970). The affinity of each of these approaches is that they focus on the activity that occurs in the laboratory or in the field and they model the scientific enterprise in general after a certain image they have of physics (Habermas). And, much like the earlier positivists, they discount the place of values in science, as anything but motivational.1

4Because historically positivists tend to see physics as the quintessential science it has developed a reputation for preferring physics-like research. Controlled field experiments along with statistical analysis and probability statements are thought to be as close as one can get to the certainty of physics. Economics and some forms of psychology such as behaviorism have been favored as close to the real thing, while much of anthropology has been dismissed as not scientific enough. Yet this is somewhat arbitrary, and much that appears as scientific in the social science has a peculiar self-confirming quality (Chomsky). Some are concerned that to take physics as the model of science and apply it to the human sciences must leave out much that is unique to human beings.

III. Positivism and Pragmatism on Science

5As a way of understanding science, positivism has a number of competitors, of which pragmatism is one. Yet pragmatism would find much that is useful in randomized field study and quantitative analysis. It is useful to recall that John Dewey often saw the social sciences as an immature science where he saw the natural sciences as already developed (LW 12). While this observation could have many different implications it is most likely that a pragmatist like Dewey would applaud the appropriate use of randomized field experiments. However, he would contextualize them within a value infused understanding of the world.

6The quarrel between positivism and pragmatism is not over the usefulness of certain methods. It is about whether it is reasonable to determine the appropriate method independently of the problem it intends to address. For someone like Dewey, positivists are too restrictive in this matter and too willing to prescribe before hand what can and what cannot be counted as good science (LW 13).

7An important element of that quarrel is about the role of values in research and whether values claims can be addressed in rational and objective ways. For the pragmatist the positivist’s understanding of value claims as meaningless or as ejaculatory utterances is a big mistake. As Dewey nicely points out, even an ejaculatory utterance like child’s cry has meaning and demands a response. Hence it cannot simply be dismissed by reclassifying as nonsense (LW 13). To put this somewhat differently, a positivist and a pragmatist may find themselves at some stage of a project recommending the use of a randomized field study. However, they will likely differ, as I will show shortly, over what should come before and after the randomized aspect of the study, in how the terms are defined and how the findings are reported.

IV. Values and the Relationship Between the Researcher and the Researched

8To be anti-positivist in the pragmatic sense is then not to be against field studies or measurement, or zealously for qualitative research. It is simply to calibrate the research enterprise in a different way by being more open to the interplay between valuing and knowing, between reporting and measuring. This involves a self-reflective understanding of the way different ways of reporting data can influence subsequent behavior.

9The image of the relationship between researcher and researched, or in Dewey’s terms, the knower and the known (LW 16), is not one of distance and detachment where the knower, as Nagel describes it, surveys the known from the point of view of nowhere (Nagel 1986), and where description has no influence on the behavior of the described. For the pragmatist this self-reflection about values is a critical part of the research enterprise, especially where social research is concerned. Knower and the known are interconnected, and values influence the definition of initial concepts, the methods selected to investigate problems and the language used to report results. A community of inquirers is important to pragmatists not just to verify conclusions but also to enable researchers to become self-conscious about their own values and how they influence the research process. Historically, pragmatists like James and Dewey have embraced the potential of a variety of methods of inquiry.

V. An Example of the Implications of Positivism and Pragmatism for an Important Contemporary Educational Issue

  • 2 To the extent that the policy was based on an assumption that closing schools would improve test sc (...)

10A few years ago the city of Chicago initiated a policy that requires public schools be closed if their students tests scores fall below a set minimum for a certain number of years. When they are closed the students are sent to other schools with different teachers and administrators. This policy is in line with the Federal guidelines mandated in the No Child Left Behind legislation first initiated during the Bush administration. The assumption driving this policy is that poor test scores mean poor teaching and that poor teaching signals inadequate administration (de la Torre & Gwynne 2009).2 However there is an additional story that needs to be told.

11As more schools closed, student violence, including murder, increased. Local activists, including some teachers, believed that there was a connection between the closing of schools and the increase in violence. They pointed out that the effect of the policy was to mix together students from rival gangs in the same schools and it required that more students walk to school and cross rival gang territory. The increase in violence has not just impacting gang members. Non-gang members and even honor students have been attacked and some killed. These activists believe that the initial policy wrongly assumed poor tests scores necessarily means poor schools, and that the side effects of school closing were never fully thought out.

12The activist’s concern is about policy, but it has even wider implications about the conception of research that supports the policy. Given that statistical studies have alerted the community to the increase in the murder rate, the problem is not with statistics as such, but with the way values were incorporated into the research project to begin with and with the failure to consider local definitions of a “good school.” Most likely safety would have been quite high on any local person’s list of the qualities of a good school.

13For the pragmatist the definition of a good research must take into account different kinds of values and different levels of understandings, local as well as bureaucratic ones. Unlike some critics of testing and randomized controlled experiments as such pragmatists would not a-priori reject the use of standardized tests in large school systems. Many pragmatists would allow that well designed tests could serve as one important reporting tool among others. However, the pragmatist would be reluctant to allow tests scores alone to define what it means to be a good or a failing school. Pragmatism’s commitment to a communal approach would lead it to endorse a consultative research process to define and address educational needs.

14For the pragmatists both local and expert understandings are important. Local agents add depth to the understanding a particular school and its environment; experts provides context showing, for example, how one school compares to others in certain kinds of environments. Chicago is now working to incorporate these two forms of understanding as experts work to identify more closely students who are at risk for violent engagement and as community members employ their understanding to try to reduce that risk.

VI. The Calibration of Facts and Values in Research

15The radical fact/value dichotomy proposed by positivism is problematic for the pragmatist to the extent that it dismisses discussions about values as meaningless, or to the extent that it allows some values to dominate the discourse by passing as “facts.”

  • 3 The pragmatist would understand that background understanding would determine the legitimacy of the (...)

16Consider, for example, the difference between a research question that asks about the death rate in Nanking in 1938 and another that asks about the murder rate in the same place during the same time. Researchers can answer both questions in ways that accurately meet the positivist standard, but one requires a much fuller understanding of the situation than the other.3

  • 4 My appreciation to for some of this wording.

17The real danger of the positivist fact value/dichotomy is not that it rids education of values, but that it creates a value vacuum, which is likely to be filled by unquestioned procedures that automatically determine the fate of a school community. The effect is to reproduce the values of the most dominant groups, and help their values masquerade as facts.4 An example would be the way IQ tests are normed to produce a Bell Curve and thereby create tests that always rank people as smart and stupid according to that same curve, albeit now expressed in numbers, hence reinforcing the impression that intelligence is a linear, one dimensional product.

18To be against positivism in a pragmatic way is not to be against testing and measurement. It is to be concerned about narrowly calibrating the researchers’ radar so that some reasonable concerns are excluded as “just opinions” and others are allowed to pass by as hard “fact,” subject neither to challenge nor inquiry. In contrast Pragmatism, especially of the more critical kind, allows that fact are vested with values and potentially can always be deconstructed and the values they conceal can be unveiled and re-evaluated in light of new considerations.

19For the positivist good educational research requires that researchers restrict themselves to what are assumed to be testable tasks, such as determining ways to increase the rate of growth of human capital development. Under this conception philosophy is limited to policing research so as to determine in this narrow sense whether or not it is scientific. If its claim can be empirically tested it is scientific, and it is “good science” if the conclusion are verified or falsified by the well-structured experiments or observations. Here the critical distinction is believed to be between the refined knowledge of the expert and the unrefined understandings and values of local actors. But this distinction can create problems.

20For the pragmatist the problem with “experts” occurs when they assume that they know the problem better and in the same way as those experiencing it. For the pragmatist the subject has a special relation to the problem that should not be ignored. To take a simple example, a person knows his or her own tooth ache in a special way even if she does not know what caused it or what can cure it. For the expert to dismiss this knowledge as unimportant is to dismiss the subject as emotionally impulsive, narrowly self-interested or conflicted with other subjects. This in turn creates a view for the subject of the experts as cold, unengaged, and ignorant about local needs and concerns. For the pragmatist, given these different relation to the experience there is a need for collaboration in defining a problematic situation and addressing it.

21For the pragmatist good educational research arises out of human needs and serves to improve the conditions of real people. One of the tests of good research is whether it takes into account a wide range of values and whether it ultimately serves both to improve the situations of people and to provide them with the intellectual tools to help them reflect on their own interests and to address their own future need. This entails the following starting points:
1. Value claims are not meaningless.
2. Local understanding may be incomplete but not inherently defective.
3. Conflicting value claims can productively be viewed as an invitation to a conversation and the beginning of an inquiry.
4. Expert knowledge can enhance that inquiry.
5. Education as the transfer of the means for continuing growth and development is at the heart of good educational research.
6. The inquiry process should leave those affected in a better condition to handle their own future difficulties.

22While pragmatism acknowledges that specialists have developed methods of refining knowledge, it also allows that local actor enjoy a privileged position in terms of the depth of experience. Yet without the other both may lack the conceptual tools required for changing the situation for the better. Local actors may have the insights but lack the wider perspective and the tools that long lasting change requires. Experts may have the tools but lack the local insight needed to apply them effectively.

23Specialized methods can be especially helpful when local understandings are unclear, when conflicting interests block further inquiry, or when there is an inadequate understanding of the wider context or historical factors that favor one view over others. They can also be helpful when conventional power and status gets in the way of inquiry. In these cases experts can be helpful in opening up paths for new experiences by generating new understandings. Local understandings are critical, however, in locating problematic situations, developing working hypothesis, defining initial terms and judging the adequacy of general solutions for individual cases. They are also important in exposing the prejudice or unacknowledged interests of the expert.

VII. Two other Alternatives: Absolutism and Relativism

24Two other alternatives vie with pragmatism in the modern world. The first is absolutism as represented in the richly textured, highly nuanced, but often confused work of Alasdair MacIntyre who seeks resolution to value conflicts by appealing to the Aristotelian and neo-Thomism traditions. The second is neo-relativism often associated with post-modernism. MacIntyre (MacIntyre 1981) rejects positivism and allows, with pragmatism, that values are to be taken seriously. He holds that value expressions are meaningful because they are connected to a certain kind practice like science, art, sports, family life, etc, and as such, they are verifiable. For example, given a knowledge of a practice, say like basketball, we have little trouble identifying, a good defense from a mediocre or poor ones. Hence, once we understand a practice and the tradition in which it is embedded, we should have little difficulty appraising value judgments about that practice. However, although MacIntyre often implies that some traditions are more worthy than others, he offers few tools through which a tradition can be critically examined, save through its capacity to resolve internal contradictions. Yet, very complex, rich traditions are likely to have significant contradictions where as some simpler, yet shallow traditions may be free of contradictions.

25MacIntyre’s model depends on a radical separation between different traditions, and only when contradictions can no longer be resolved can a tradition be evaluated. In contrast pragmatism is willing to acknowledge that the world is often messy and solutions to problems may be partial, at best. MacIntyre’s is a neat and tidy world where we should always know, at least in theory, just what practice we are engaged in, what tradition it is apart of and what standards should be used in judging its performance. In real life, however, there is often uncertainty about what kind of practice we are engaged in and what tradition it belongs. Pragmatism is able to acknowledge this fact: MacIntyre is not.

26Any parent who has had to confront a conflict between the responsibility to their children and their responsibility to their job runs into the same issue on a personal level. What practice am I engaged in – worker or parent? And to what tradition do I belong? Under one tradition a mother should not even be working and thus should feel extra guilt if she has a job outside the home. Under another, as a woman she should have the same right to employment as a man and thus is justified in feeling extra resentment if she is denied this right just because she is a mother. MacIntyre writes as if he has answers to these predicaments, but he does not. He simply fails to acknowledge the push and pull of different roles and the overlapping of different traditions and presupposes a coherent, settled, isolated tradition as the norm.

  • 5 It is worth noting that traditional positivism draws on both with analytic truth having the charact (...)

27The failure to provide a convincing account of absolutism is relevant for the assessment of pragmatism. Because pragmatists reject absolutism in its various forms, it is thought that it must embrace an arbitrary relativism where power and wit determine what can pass for truth. The critics however fail to distinguish here between truth with a small “t” and Absolute Truth with a capital “T” where the term carries with it a kind of divinely guaranteed certainty.5

28For the pragmatist a true claim is one that stands up to rigorous tests and has the status of a warranted assertion. Certainly some future test may come along that de-warrants the assertion, but this only calls into question whether any reasonably justified claim to truth is immune from modification. It does not equate warranted claims with arbitrary claims that people accept simply because they are forced or tricked into doing so. Indeed, one of the main goals of Dewey’s philosophy of education and its emphasis on science and experience was to create a population that would guard against the acceptance of arbitrary claims. Sometime neo-pragmatists like Richard Rorty have fueled this criticism with quips like: “Truth is what our peers will, ceteris paribus, let us get away with saying” (quoted in Bredo 2009: 442), but the broader import of this for most pragmatists, including Rorty, is that our understanding of truth is always subject to revision.

29For the pragmatist the emphasis is on truth (small “t”) as an instrument for engaging the world. We understand the limits of our truth claims not just through the way others respond but also as the world pushed back to tell them and us “you do not have it quite right.” Other people are an important part of this but they are not the only part. Often when they reject our truth claims it is because they quite rightly see just how firmly the physical and social environments are pushing back. Rorty does not miss this point, and when he does acknowledge it, his pragmatism seems in evidence.

30Experience, not guile, is the arbiter. For Dewey and for Peirce when the world pushes back it helps us to decide whether a belief should continue to serve as a guide to action. This is why for Dewey, “true” and “false” are not quite the right terms. Rather a claim is “warranted” or not depending on whether it seems a sound guide for action. Whatever our motives for affirming a claim, it may be status, stubbornness or power, the verification of the claim – however tentative – will depend on the evidence available to support it, and the role it plays in developing possibilities for new experience. T (t)ruth, including moral truth, is prospective for pragmatists and grows.

VIII. Moral Invention: An example

31There are many examples of the way in which moral knowledge grows but the debate over euthanasia can serve as a brief example. Here one side holds that mercy killing is murder, clear and simple while the other believes that intense and chronic suffering justifies aiding a person in accomplishing a self-willed death and that not to allow it is simple cruelty. Some traditions, including Catholicism, have helped lay the ground work for a partial reconciliation of these views by drawing a distinction between taking active steps to kill someone, such as administering poison, on the one hand, and letting a person die, by say removing life support systems, on the other. While this moral invention certainly does not solve all the issues regarding mercy killing, it does help in those cases where patients need life support systems if they are to continue living.

32Another moral invention along this line has been developed in response to an ever-improving capacity to enable vital organs such as the heart and lung to continue functioning by mechanical means. Here a refinement of the concept of death allows that a person may be declared dead if there is no brain activity even if other vital organs, such as the lungs and heart are still functioning. In these cases the pragmatist joins with the absolutist in seeking a resolution that is respectful of the traditional view, but she joins with the relativist in providing more flexible interpretations of established dogma. Some times these innovations come from within a tradition as, for example, when the Catholic Church decided that not all lending was usury and redefined the concept as human exploitation in general (Feinberg 2006). At other time the innovation may be initiated from outside a tradition in response to some general need. A good moral invention is a way of resolving the absolutism of tradition with the flexibility of relativism. Of course invention does not always end controversy, but it points in a productive direction that allows apparently dead ends to be reviewed for possible paths of escape.

IX. The Pragmatism and Values: Or, How Does the Chicken Get Across the Road?

33There is a famous paradox by the ancient Greek philosopher Zeno that describes the plight of a chicken that is trying to cross a road. According to Zeno, before the chicken can get across the road she must cross the half way point and before reaching the half way point, she must cross the half way point to that point and so on. Since there is always a new halfway point the only conclusion that we can draw is that the chicken could never begin to cross the road. Now, ask a pragmatist to solve the paradox and the answer to how the chicken could even cross the road likely will be a rather simple: “one step at a time and with an occasional forward flutter.”

34Just as this response dismisses the problem as a mere exercise for formal logicians and beginning philosophy students to puzzle about, the pragmatist would provide a similar response to the positivists understanding of value claims. Chickens have no problem crossing roads (assuming no traffic) and people have no problem understanding and coming to terms with value claims, assuming sufficient information and an openness to the possibility that, under certain circumstances, they may have to revise their own standpoint.

35The interesting things about values are the ways people argue about them, explain why they believe one to be better than another, and arrange their lives according to them. Not only that. They also test their goodness. There’s a prospective aspect here that checks desire as, for example, when someone asks: “Is my desire for cigarettes good for me?” Surely value claims are not meaningless. People fight, negotiate, compromise over and cooperate through them all the time. So, if positivists can’t find an appropriate way to understand the significance of values other than dismissing them as a modern version of Zeno’s paradox, then so much the worse for positivism.

36The pragmatist is more in tune with intuition and everyday practice than the positivist, more open to judgment of better and worse than the relativist, and more willing to eschew the ideal for the acceptable than the absolutist. In the abstract of course there is nothing about the logic of this response that must convince the positivist, except that it speaks to life; not simply to logic.

37Pragmatism rejects the fact/value dichotomy as an artificial ontological distinction. Facts without human interest do not have a reality of their own. Facts are the outcomes of inquiries. No inquirers, no facts. And values do not exist independent of the means to realize them. Hence, it is an illusion to speak of either fact or value independent of human interest and it is also an illusion to hold the view of many religions that there is some ultimate end to human life above and beyond the strivings of human beings.

38Essences, even in MacIntyre’s watered down formulation as virtues inherent in a practice, simply do not exist, except as constructed through and by human experience and reflection. Virtue is a socially approved quality of human activity as refined over time; and “tradition” is simply a shorthand way of describing the codification of these virtues across generations of human beings. To reify tradition in the way that MacIntyre does is to obscure the fact that any one person may shape their life within, across or beyond recognizable traditions. It also leads us to look backwards rather than forwards and to assume that the tried and true is always best even when new experiences are encountered. In doing so MacIntyre’s absolutism, ignores the fact that some forms of conduct viewed now as virtuous may be subject to radical revision in the future.

39In contrast, for the pragmatist, the requirements of even a simple virtue like honesty change depending on circumstances. Honesty requires one thing when, say, deciding whether to tell the clerk he has given you too much change when you are right in the store. It requires another if you notice the small amount of extra change after driving miles to your home. A virtue like honesty also requires discretion in and when it should be applied. If your uncle tells you that he really wants your honest opinion about the poem he has just composed it may not always be the best course to tell him that you think it should be shredded and burned, even though you do think it should be shredded and burned.

X. Values in Science

40Pragmatists reject the radical positivist idea of value-free science and point to various ways values interpenetrate science. On a practical level value issues are present in critical debates about what kind of science will be funded. Will it be big science that requires billions of dollars to build say a huge particle accelerator, or small science that requires say a few thousand dollars to investigate many different projects, say like the health issues in fast food consumption?

41On the theoretical level values are involved in decisions about which theories to adopt. For example at the time in which the heliocentric theory of the universe was first proposed and until Newton developed his theory of gravitation, there were serious problems that the heliocentric theory could not adequately address. Perhaps the most embarrassing was the question of why the earth did not lose its atmosphere as it traveled around the sun? This problem simply did not exist under the earth-centered theory where the earth stood still. Moreover, the earth-centered theory was able to explain many of the successful predictions first proposed by the heliocentric one. Ultimately the latter was accepted not just because of Newton’s answer to the puzzle about the atmosphere, or because the heliocentric theory could not provide explanations for the newly observed celestial phenomena, but also because it was a simpler theory, one that was more aesthetically pleasing. It also proved more productive as well. This meant that it was able to generate new and confirmable predictions and not just to explain them after they were confirmed.

42Values also enter into science whenever a decision is made about how to define or classify something. Water is H2O only when it enters into experiments or other similar activity. Otherwise it is a thirst quencher, vodka chaser, or drought stopper. Values are even more obviously implicated in social “facts.” It makes a big difference whether the study of group behavior is called a “crowd” or a “mob.” These difference may go quite deep into the foundational logic of science, complicating something as fundamental as the identity principle, A=A. Take the following example, which counters both the logic of identity and the positivist’s definition of analytic as true by virtue of definition
Premise:
Major premise: all bachelors are unmarried (true by definition).
Minor premise: all the men in car A are married (empirical claim).
Identity qualifier: all men in car A are the same men as in car B.
Question: are all the men in car B married?
Answer: no, because car A was in Massachusetts, which recognizes same sex marriage, and car B crossed the state line into Road Island where same sex marriage is not recognized. Now clearly clever logicians can neaten this so that the apparent inconsistency disappears, but in doing so they must acknowledge the value-laden feature of a factual claim like “there are six married men in this car.”

XI. On the Objectivity of Values

  • 6 Granted this requires a commonsense understanding of “heart and mind” “in-here,” and “subjective” w (...)
  • 7 These are my terms.

43One of the intuitive appeals of the positivist understanding of the relationship between facts and values is the view that facts are simply out there to be discovered and therefore are essentially objective where as values are somewhere in-here – in my heart or in my mind – and are thus inherently subjective.6 Further it is thought that when we decide something on the basis of the facts of the case we are being fair, where as when we decide matters on the basis of values we are deciding matters subjectively and hence we are arbitrary. Hilary Putnam a modern pragmatist rejects these associations (Putnam 2002). First, he rejects the out-there-ness of facts and the in-here-ness of values.7 Second, he rejects the essential objectivity of facts and the essential subjectivity of values and third he rejects the distinction drawn between factual judgments as fair and value judgments as arbitrary.

  • 8 An example of the rejection of potentially useful research findings occurred after WW II when the a (...)

44Because Putnam’s is one the most logically sophisticated contemporary pragmatist, it is worth spending a moment examining his position. Putnam lists some of the value norms that go into deciding the worth of a scientific theory. These include: epistemological norms used in judging the merit of scientific theory such as coherence, consistency and the like; aesthetic norms such as simplicity, the beauty of theory and its internal perfection. There are also moral norms that determine whether a scientific experiment is worth performing no matter how much knowledge it yields.8

45Social science often conceal moral judgment by re-labeling common sense terms. Intelligence becomes “IQ,” punishment becomes “negative reinforcement” and reward is transformed into “positive reinforcement.” To the extent that this relabeling allows for a reasonable and systematic reconstruction of common sense understandings it can be very important. However, to the extent that it rejects common sense experience it can be destructive.

  • 9 Such change in standards is often a theme of literary works. See, for example, Coetze 2000, a book (...)

46For Dewey and Putnam labels such as “cruel,” “just,” or “brave” have an objective standing in that they appeal to evaluative standards as developed and shaped through the needs and common sense understanding of a community. In that sense an observer could take a sample of behaviors that the community labels brave and provide a reasonable reconstruction of what these standards are and how they are applied. Moreover judgments of standards change depending upon changes in the environmental, the social conditions, and are influenced by the development of new knowledge and understandings an insight which is at the foundation of many important novels (Coetze 2000; Ishiguro 1989).9

47One of the mistakes the positivist makes involves an unstated assumption that when something is rational and objective it must result in agreement (Dewey himself sometimes fell into this position, especially when extolling the virtues of “the scientific method”). This is the basis of the positivist’s mistrust of value disputes as meaningless.

48For Putnam value differences and moral commitments are rational not when they insist on agreement but when they leave an opening for repudiation. For example, if someone committed to justice and equality but then always acts in ways that benefit her own race, then she needs to re-examine what she means by justice.

49For Putnam, we do not need agreement to live in same moral world but we do need to respect differences. For Dewey there was a slightly different emphasis to the objectivity of values. He believed that ultimately coordination of action – both on the individual level, as I coordinate my hand with my mouth when I eat, and on the social level, as individuals coordinate with one another in pulling on a heavy object with a rope, – would serve as a measure of value. Yet the more prominent side of Dewey is quite consistent with Putnam. For both of them values become valuable through a process of criticism and evaluation (LW 13). For both the emphasis is more on process than product, more on ends in view rather than ultimate ends.

50To the positivist the researcher is thus like a neutral umpire who just calls things as they are. For Putnam the relationship is more complex and values inform the research process at every stage. For example, values may indicate what the researcher counts as successful intervention. In medical research, for example, what to counts as a cure may be different depending on the age of the patient. For older men with prostate cancer, medicine that slows the progression rate in half may be as good as surgery which removes the prostate all together. If not a cure in some conventional sense such medicine might be preferred to surgery, considering side effects and estimated longevity. Here “cure” may be defined either in terms of eliminating the cancer or improving quality of life.

51Values also enter the picture in determining the appropriate restraints on scientific studies. The famous Milgram (Milgram 1963) experiments at Yale that tested people’s willingness to obey authority, even when doing so went against their own conscience, came under heavy criticism for deception and for placing the subjects under sever stress. This criticism was one of the motivations for the development of a new moral invention, the requirement of informed consent, where researchers have to explain the level of danger or stress that subjects might experience and get their written consent to perform the experiment.

  • 10 My appreciation to Fred Lighthall who is writing a book on this episode.

52In addition values enter into scientific work in decisions about threshold levels. For example, one of the factors involved in the Challenger space probe failure which killed all aboard was the way the company, Morton Thiokol, in cooperation with the space agency set the burden of proof. At the time, the burden was placed on those engineers who thought a launch was unsafe. They had to prove their case. Since the O rings had never been tested under the exceptionally cold temperature expected for the launch no one could say for certain what would happen to them, and so the mission was launched. An alternative threshold would have required the burden of proof be placed on those who thought that it was safe to launch. If this could not be shown then the mission would have to have been postponed but lives saved (Davis 1998).10

XII. Neo Positivists and Pragmatists on the Calibration of the Fact/Value Continuum

53Neo positivist would likely have little problem acknowledging that values do in fact play these roles in research but would then argue that there is a line between science and non-science and when values enter the picture the line has been crossed. However, for the pragmatist, this is less a problem of demarcation, as the post-positivists would call it, and more an issue of calibration. The model advanced by positivists calibrates the research enterprise in a way that defocuses attention from value concerns as if they were not really important for the conduct of science. Questions like “who is framing the problem?” and “who is defining the terms?” are not easily placed inside the post-positivist’s radar, and hence there is little inclination to examine the goodness of the initial frames and definitions. This results in part from equating rational deliberation with empirical, testable studies, and then joining with the relativist in allowing that all else is opinion. Of course there is a legitimate concern behind this, one shared by pragmatist as well, that the researcher’s private values not drive the findings of the research. There is good reason to distrust the tobacco company “scientists” who, on the basis of a missing chemical link between smoking and cancer, declared cigarettes had not been shown to be unsafe.

54For the pragmatist the argument about whether values belong in science is unproductive. Rather research needs to be calibration so as to provide room for exploring the implications for human well being. The researcher should be open to inquiring into the consequences of a certain way of framing a problem and to the benefits one or another way of framing can provide for different groups. The fact that this calibration entails valuation does not require that values override science, say in some Lysenko – like program. It simply means that pragmatism opens up value claims to rational deliberation.

55While pragmatism is friendly to experimental and statistical research, it calibrates its idea of good research in a way that can capture the unstated values that implicitly drive the conducting and reporting of research findings. And part of the job of pragmatism is to then open up these values for inspection and to engage people in discussions about them. That specialized experts have a crucial role in research goes without saying (LW 2), but the fact that values are imbedded in different aspects of a research project means that there is considerable room for local actors to interrogate the findings. One of the functions of scientific and professional education for the critical pragmatist is to teach researchers how to be mindful of value issues and how to engage the public in productive discussions regarding the value implications of their work.

56Pragmatism certainly endorses the usefulness of statistical and experimental research. For example this research has been invaluable in understanding the importance of class size on the improvement of reading in the United States (Mosteller & Boruch 2002). Yet when used alone and without the insight of practitioners it is limited in understanding why this is the case. In some instances these factors may be fairly obvious. Smaller classes mean that teachers can spend more time with each student and can isolate and treat his or her specific problem. Sometime this may be quite simple. A youngster who had trouble keeping up with the class in singing from a written songbook may have only to be shown that in Western music the convention of reading stanzas of songs differs from the convention of reading lines in a storybook.

57Sometimes a problematic educational situation is more complex, as revealed for example in the studies of Ray McDermott (McDermott 1993) who, through hours of recording classrooms and observing video tapes, shows how a teacher and the poorest reader in the class unconsciously work together in a kind of dance that assures the student is never embarrassed by being called upon to read, but also assures that she is never taught how to read. As a pragmatist would point out, tests may be useful in helping to determine the reading level of a child, but McDermott’s research shows other issues that may need to be addressed if reading is to improve. For a pragmatist research methods are tools. The best methods are those most likely to help understand and address the problem at hand.

58One of the important contributions of pragmatism is to connect science to common sense to refined methods of inquiry. As Dewey wrote in his Logic: “Scientific subject matter and procedures grow out of the direct problems and methods of common sense, of practical uses and enjoyments, and react into the latter in a way that enormously refines, expands and liberates the contents and the agencies at the disposal of common sense” (LW 2: 71-2). Science then adds to common sense by opening up new ways to understand relationships and possibilities. It expands judgment based on previously constricted experience and provides new tools that aid thought and action.

59Consider, for example, the simple but obvious ways in which common sense grows. Take the historical evolution of a common sense of direction from pointing; to “here/there”; to “left/right/front/back”; to “North/South/East/West”; to “degrees of longitude and latitude”; to the technology of triangulation used in global positioning systems or the directional instruments and concepts used in space travel. Each stage has expanded the possibilities for navigation and an early advance becomes the common sense of a new one. As science moves beyond the latest innovative sense of direction and it gets re-incorporated back into a new and expanded common sense of direction. The old sense is not discarded – we still point and we still say “here” and “there” – but is rather augmented. Even astronauts in space with the most advanced navigational equipment at their fingertips will still point to a wrench and say “See that over there. Could you bring it here?” The big insight of pragmatism then is that science creates new conceptual and technical instruments that can then serve to liberate common sense not trespass over it.

XIII. The Contribution of Critical Pragmatism

60Critical pragmatism allows that everyday understanding is sometimes inadequate in defining a situation as problematic, especially in cases where power or experience is unequal. Here common sense may simply accept the situation as a fact of life. In these cases critical pragmatism encourages a dialogue between refined research, and every day understanding about the systematic silences that often mark subordinate or oppressed status.

61Critical pragmatism thus supplements traditional pragmatism by highlighting those situations of inequality where local understandings may be systematically silenced or unrecognized. This means a greater sensitivity to the historical relations between groups where unequal power or experience results in the domination and systematic silencing of the one by the other. Here the primary need is not to resolve a predefined problematic situation – in Dewey’s terms – but rather to provide discursive structures that will give voice to the dominated group, thereby enabling its members to identify certain situations as problematic. For example, feminist researchers have observed classes in which girls are called upon much less frequently than boys, partly because boys raise their hands more often and more vigorously than girls. One practical suggestion has been for teachers to wait longer before calling on a student and to encourage girls to speak out.

62For the critical pragmatist the goal is not to police research so as to purify it from value claims, as the positivist would do. Nor is it just to link research to common sense understanding. It is to also give expression to those private, isolated, serially undergone experiences of marginalized group members by exploring the historical inequities that render them publicly inexpressible and unrecognized. This requires familiarity with the conditions that silence some people as well as of the potential avenues and organizations that can give them voice. Critical pragmatism is aligned with traditional pragmatism and especially with the ideas of Dewey, but whereas Dewey starts inquiry with recognition of a problematic situation the critical pragmatism may begin an inquiry with the awareness that oppressive social structures can sometimes silence the expression of values or render their expression incoherent or inappropriate. This was often the case with adolescents where their emerging individual sexuality was often unacknowledged by schools, leaving many teenagers to feel isolated and perverse.

63The awareness of the unarticulated aspects of problematic situations brings with it a consciousness of the significance of subgroup identities, or, of groups of people, whether minorities, teenagers, women, gays or bisexuals, who share a common yet unarticulated needs. This recognition is one of the important distinguishing factors between traditional and critical pragmatism.

64Historically in the United States much of the traditional pragmatist’s early educational efforts involved the inclusion of individuals through assimilation and an educational process that sometimes involved disengaging the individual from a subgroup identity (Feinberg 1975). Today much of the effort of critical pragmatism involves increasing the recognition of oppressed groups and their members, with considerably less emphasis on assimilation (Glaude Jr.). Nevertheless, in contrast to the positivists who tend to dismiss value claims as meaningless or redundant, pragmatists, both traditional and critical, see differences in value utterances and conflicts as an invitation to a conversation and inquiry. In this sense, value differences for the pragmatist are not to be dismissed as matters of opinion with the conclusion that they are not subject to inquiry, but as moments for pause, and humility in the awareness that other forms of life and other modes of reason are valued.

65Critical pragmatism is concerned with repressed needs and silences resulting from systematic, long-standing and severe inequalities. Like traditional pragmatism it acknowledges the link between local experience and refined knowledge but it takes one step back from the traditional pragmatist and wants to know how each is constructed, especially in situations of systematic, historically generated political, social and economic inequality.

66Critical Pragmatism allows that there are situations where inequality has been so ubiquitous that it has been bred into common sense understandings at all levels and that here there is a strong disconnect between the local experience of inequality and its local expression. In these cases, critical pragmatism looks for the silences that blocks expression and analyzes the history of common sense and its construction in order to understand how it serves to perpetuate systematic inequality. In these cases philosophy can help educators become aware of ways that they unreflectively endorse values of inequality. And it can help researchers understand how they can serve to develop new understandings. Its goal is to enhance critical reflection within the arena of common sense, and to use this understanding to develop new research projects.

  • 11 I use the term suspicion here to indicate a certain response to modernity, and, in Bourdieu’s case, (...)

67Critical pragmatism needs to be distinguished from critical theory in its various Continental forms. It does not assume, for example the hermeneutics of suspicion, typical of the French theorists like Foucault (Foucault 1965, 1970) or Bourdieu (Bourdieu & Passeron 1977);11 nor does it assume a priori that one form of ideal discourse fits all, as with Habermas (Habermas 1968). Critical pragmatism is open to whether there is a problem that needs to be addressed and it is also open regarding what might count as a reasonable resolution of that problem. It does not begin by assuming, as dogmatic followers of Freire do, that oppression is at the basis of all educational differences, although sometimes oppression is indeed the critical factor. Nor does it begin with the belief that all problems can be reduced to distortions in communication, although this too is sometimes the source of the problem. Nor, on the other side, does it assume that the problem must lie with unmotivated students, incompetent teachers or unresponsive bureaucrats. Yet to find out just where the problem and its source(s) lie, it must often interrupt common sense and the self-understanding that goes along with it.

XIV. Interrupting Common Sense Logic

68Common sense is the shared understanding that peoples have of everyday situations, the default logic appropriated to support it, and the judgments that issue from it. Often common sense is articulated in brief exchanges that virtually everyone accepts as true, and that at the same time reinforces the conditions required to sustain it as true. One example would be girls who were discouraged from becoming doctors, until the feminist movement began to challenge certain stereotypes.

69Very often this was done with the best interest of the student in mind and with an eye to “reality” as defined by the existing situation. Women were not doctor. Few applied and few medical schools accepted them. Some girls who might have wished to be medical doctors bowed to reality and altered their expectations to fit it. The common sense understanding here was that women could not enter medicine and that few girls would want to do so anyway.

70Today, when there is about an equal number of women and men graduating from medical school in the United States, the lie has been given to that common sense understanding. To change the earlier situation the critical pragmatist would not have social science build on existing common sense but would need to critically deconstruct it, showing how the mutually supporting standpoints result in perpetuating an existing but taken for granted inequality. Consider the following example that would have been common up until a couple of decades ago:
Girl: I want to be a doctor, what courses do I need to take?
Middle school teacher: There are good careers in nursing or occupational therapy. I would not advise medicine.
Girl to parents: Ms Jones says I should be a nurse rather than a doctor
Parent: that is a good idea
Girl: Ok, I wonder what I have to do to become a nurse.

71The decision of the girl to change career goal would have been practical one, given a reality constructed through existing common sense, and the fact that medical colleges were reluctant to admit women and few women practiced medicine. However, given the fact that medicine was, for all practical purposes, closed to talented women existing common sense was also a violation of a basic principle of democracy, i.e. equal opportunity. Given too that certain medical conditions are experienced exclusively or more commonly by women, and that male doctors and researchers were often insensitive to them, existing common sense also had the real consequence of limiting understanding on female health issues.

72Nevertheless there was a very clear logic that supported the existing situation. The logic was predicated on a strong division of labor where women were supposed to marry and raise children and men were supposed to work outside the home and have careers.

73Medical schools hence felt that an expensive medical education for women would be a poor long-term social investment. And woman who looked at medical schools found that indeed, almost all the students were men. So what was the use of applying? And if one was not going to apply why take all those chemistry and math courses, etc.

74Given these expectations a certain reinforcing logic developed with the result that few women applied to medical school and very, very few were accepted. From the standpoint of any key stake holder in the process this result was not only sensible, in the way in which common sense is sensible, it was challenged very infrequently and mostly in private. It made sense because of a series of interlocking syllogisms, where, whether accurate or not, one could have the best interest of girls in mind and still support the existing situation. Yet because the situation is self reinforcing, because it violates the ideal of equal opportunity, because it risks poorer health care for women, and because the doctor/nurse relationship mirrors the dominance of men over women, there was good reason to interrupt this reinforcing logic and explore the openings for change.

75Intervention involved both understanding the different common sense logics and some of the more subtle ways in which they mutually reinforced each other. Suppose, for example, the intervention on behalf of women was to be led by a male doctor. Here the very relation of dominance that the reform was intended to challenge would be reinforced. Hence, critical pragmatism must not only be aware of the goodness of a reform and the common sense logic that may retard it, it also must be aware of the process used to implement it and to determine its direction. In many cases this means that sympathetic members of a dominant group must step to the side, providing needed service but not leadership, as the reform is developing.

XV. The Pragmatic Temperament

76In reconnecting science and values it is useful to think of pragmatism not as a set of fixed principles, but as a certain kind of temperament. This pragmatic temperament calibrates the scope of rational discourse and appraisal to includes to include values and, in contrast to positivism, seeks to connect science to common sense as it works to refine each of them. To do this, the pragmatic temperament is skeptical of the positivist claim that values are simply individual preferences and instead it understands differences about values as an invitation to a conversation. This invitation holds out the possibility that values can be rationally considered and objectively appraised (LW 13). To say that a value claim is objective means that it meet the following conditions as suggested by Elizabeth Anderson:
• There is a sincere acknowledgement of possibility of error and possibility of changing minds, including one’s own.
• Authority to give reasons is granted to the different claimants.
• Claims by the same party do not contradict one another, and when they do it calls for critical self-reflection each party is willing to apply critical reasoning to their own proposals as well as to those of others.
• There is good faith effort to seek a common point of agreement from which justification can proceed.
• There is openness to the introduction of novel considerations as reasons (Anderson 1993).

XVI. Conclusion

77A value claim does more than simply express an opinion or a subjective desire, as the positivists believe. It directs attention: “Look at the sunset!” It makes a promise: “There is something worth seeing.” It affirms a hypothetical basis for agreement, and it proposes approaches for living together even in midst of disagreement. The pragmatic approach to value emphasizes conversation and a mutual effort to convert by reason – not by force or violence. Given its understanding of values, pragmatism views research as both value infused and value concerned. The role of the researcher is to enrich the experience of the researched. Critical pragmatism adds the requirement of critical self-reflection and the engagement of others in jointly defining the problem and closing the gap between the researcher and the researched. The pragmatic temperament when applied to education would bring local understanding in conversation with expert knowledge allowing that the latter serves as an instrument to enrich the experience of the former.

Haut de page

Bibliographie

Anderson E., (1993), Values in Ethics and Economics, Cambridge, Harvard University Press.

Ayer A. J., (1936), Language, Truth and Logic, New York, Dover.

Bourdieu P. & J. C. Passeron, (1977), Reproduction in Education, Society and Culture, London, Sage.

Bredo E., (2009), “Getting Over the Methodology Wars: Comments on Howe,” Educational Researcher, 38 (3), 441-8.

Chomsky N., (1959), “A Review of B. F. Skinner’s Verbal Behavior,” Language, 35 (1), 26-58.

Coetze J. M., (2000), Disgrace, New York, Penguin.

Davis M., (1998), Thinking Like and Engineer. Studies in the Ethics of a Profession, New York, Oxford University Press.

Dewey J., (1927), The Public and Its Problems, The Later Works of J. Dewey, 1925-53, vol. 2 (LW 2), ed. by J. A. Boydston, Carbondale and Edwardsville, Southern Illinois University Press, 1984.

Dewey J., (1938), Logic: The Theory of Inquiry, The Later Works of J. Dewey, 1925-53, vol. 12 (LW 12), ed. by J. A. Boydston, Carbondale and Edwardsville, Southern Illinois University Press, 1984.

Dewey J., (1939), Theory of Valuation, The Later Works of J. Dewey, 1925-53, vol. 13 (LW 13), ed. by J. A. Boydston, Carbondale and Edwardsville, Southern Illinois University Press, 1984.

Dewey J., (1949), Knowing and the Known, The Later Works of J. Dewey, 1925-53, vol. 16 (LW 16), ed. by J. A. Boydston, Carbondale and Edwardsville, Southern Illinois University Press, 1984.

Feinberg W., (1975), Reason and Rhetoric: The Intellectual Foundations of 20th Century Liberal Educational Policy, New York, John Wiley.

Feinberg W., (2006), For Goodness Sake, New York, Routledge, Taylor & Francis Group.

Foucault M., (1965), Madness and Civilization: A History of Insanity in the Age of Reason, New York, Vintage.

Foucault M., (1970), The Order of Things: An Archeology of the Human Sciences, New York, Vintage.

Glaude E. S. Jr., (2007), In a Shade of Blue: Pragmatism and the Politics of Black America, Chicago, The University of Chicago Press.

Habermas J., (1968), Knowledge and Human Interest, Boston, Beacon.

Ishiguro K., (1989), Remains of the Day, New York, Vintage.

Lakatos I., (1970), “Methodologies of Scientific Research Programmes,” in Lakatos I. & Musgrave A. (eds.), Criticism and the Growth of Knowledge, Cambridge, Cambridge University Press.

MacIntyre A., (1981), After Virtue, Notre Dame, Indiana, University of Notre Dame Press.

McDermott R., (1993), “The Acquisition of a Child by a Learning Disability,” in S. Chaiklin & Lave J. (eds.), Understanding Practice, New York, Cambridge University.

Milgram S., (1963), “Behavioural Study of Obedience,” Journal of Abnormal and Social Psychology, 67 (4), 371-8.

Mosteller F. & R. Boruch (eds.), (2002), Evidence Matters: Randomized Trials in Education Research, Washington DC, Brookings Institution Press.

Nagel T., (1996), The View from Nowhere, Oxford, Oxford University Press.

National Research Council, (2002), Scientific Research in Education, edited by R. J. Shavelson & L. Towne, Washington DC, National Academy Press.

Popper K., (1959), The Logic of Scientific Discovery, London, Hutchinson.

Putnam H., (2002), The Collapse of the Fact/Value Dichotomy and Other Essays, Cambridge, Harvard University Press.

Quine W. V., (1960), Word and Object, Cambridge, MIT Press.

Quine W. V., (1993), From a Logical Point of View, New York, Harper.

Torre M. & J. Gwynne, (October 2009), When Schools Close. Effects on Displaced Students in Chicago Public Schools, Chicago, Consortium on Chicago School Research at The University of Chicago Urban Education Institute.

Haut de page

Notes

1 While this may seem less true of Lakatos who allows that personal ambition may motivate the scientific enterprise, he understand values largely as nonrational motivators rather than factors in the scientific process itself.

2 To the extent that the policy was based on an assumption that closing schools would improve test scores recent survey research has shown that except for a very few students who were sent to high performing schools, most of the students who were displaced did not improve their test scores. Cf. Torre & Gwynne 2009.

3 The pragmatist would understand that background understanding would determine the legitimacy of the question and the first question could well lead to an important insight if asked by someone unfamiliar with the situation. In other words, the extreme spike in the death rate could, given an unknowing investigator, lead to questions about causation. For an already knowledgeable investigator to speak only of deaths and not murder would be a cover up.

4 My appreciation to for some of this wording.

5 It is worth noting that traditional positivism draws on both with analytic truth having the character of “T” truth (without the divine guarantee) and synthetic truth enjoying only the little “t.”

6 Granted this requires a commonsense understanding of “heart and mind” “in-here,” and “subjective” which a strict positivist would question.

7 These are my terms.

8 An example of the rejection of potentially useful research findings occurred after WW II when the allies sealed the research of the notorious Nazi Doctor, Joseph Mengele.

9 Such change in standards is often a theme of literary works. See, for example, Coetze 2000, a book that explores how a reevaluation of the colonial experience in South Africa paralyzes any action based on previous moral expectations. Also see Ishiguro 1989.

10 My appreciation to Fred Lighthall who is writing a book on this episode.

11 I use the term suspicion here to indicate a certain response to modernity, and, in Bourdieu’s case, toward the possibility of education to advance human well-being. While both are powerful instruments in shedding light on particular form of oppression in Bourdieu’s case the suspicion is build into his very definition of terms and in Foucault’s into his methodology itself.

Haut de page

Pour citer cet article

Référence électronique

Walter Feinberg, « Critical Pragmatist and the Reconnection of Science and Values in Educational Research »European Journal of Pragmatism and American Philosophy [En ligne], IV - 1 | 2012, mis en ligne le 23 juillet 2012, consulté le 28 mars 2024. URL : http://journals.openedition.org/ejpap/786 ; DOI : https://doi.org/10.4000/ejpap.786

Haut de page

Auteur

Walter Feinberg

University of Illinois
wfeinber[at]illinois.edu

Haut de page

Droits d’auteur

CC-BY-NC-ND-4.0

Le texte seul est utilisable sous licence CC BY-NC-ND 4.0. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.

Haut de page
Rechercher dans OpenEdition Search

Vous allez être redirigé vers OpenEdition Search