|
On the Conditions
Necessary for Obtaining Argument Structure Complexity Effects
|
University of
Massachusetts at Amherst
University of
Massachusetts at Amherst
University of
Massachusetts at Amherst
The research reported here was supported by
National Institutes of Health Grants HD-18708 and HD-07327 to the
University of Massachusetts and by a Mount Holyoke College Alumnae
Fellowship to A. René Schmauder.
Correspondence may be addressed to: A. René
Schmauder, Department of Psychology, University of Massachusetts at
Amherst, Amherst, Massachusetts 01003. Electronic mail may be sent to rene.schmauder@psych.umass.edu.
Shapiro and colleagues (Shapiro
& Levine, 1990; Shapiro, Zurif, & Grimshaw, 1987,
1989) have recently reported that a cross-modal lexical
decision (CMLD) task is sensitive to the amount of thematic information
associated with the lexical entries of verbs. They consistently found
longer lexical decision (LD) times in a CMLD task when an LD probe was
presented visually at the offset of an auditorily presented verb that had
more argument structure frames (thematic information) associated with its
lexical entry compared with verbs with fewer argument structure frames. Shapiro et al. (1987, 1989) called this
the "argument structure complexity effect. "
In an attempt to determine whether the
argument structure complexity effect was generalizable to other on-line
paradigms, Schmauder (1991) ran four experiments,
including one using the CMLD paradigm of Shapiro et al.
(1987). In none of these experiments were argument structure complexity
effects evident, so Schmauder concluded that the effect might not
generalize easily.
More recently, Shapiro,
Brookins, Gordon, and Nagel (1991) reported results replicating the
argument structure complexity effect reported by Shapiro et
al. (1987, 1989) and Shapiro and Levine
(1990). Furthermore, Shapiro et al. (1991) provided
evidence that Schmauder's (1991) failure to find the
argument structure effect with the CMLD paradigm was due to the nature of
the secondary LD probe items used by Schmauder: Schmauder's probes were monosyllabic
and relatively uniform in frequency and length compared with Shapiro and
colleague's probes, which were multisyllabic and less uniform in frequency
and length. Shapiro et al. (1991) concluded that
Schmauder's probes permitted "subjects to perform the secondary task
with little effort" (p. 993) and that the LD task "must be made
difficult enough to yield significant interference, allowing thematic
effects to be exposed" (p. 993).
The present observation reports the results
of an experiment that failed to exhibit an effect of verbs' argument
structure complexity (thematic complexity) with secondary LD probes used by
Shapiro et al. (1987). First, we present a description of
differences in materials across experiments. As noted above, the first to
report the argument structure complexity effect were Shapiro
et al. (1987) using a CMLD paradigm. Five verb categories were used:
(a) transitives, (b) nonalternating datives, (c) alternating datives, (d)
two-complements, and (e) four-complements (see Shapiro et al.
, 1987, or Schmauder, 1991, for a detailed
description of differences between the verb categories). An argument
structure complexity effect would result in LD times to targets presented
after four-complement verbs being longest and LD times to targets presented
after transitive verbs being shortest. Sentences in which these verbs were
embedded consisted of a subject noun phrase (NP) modified either by a
relative clause or by a prepositional phrase (PP), followed first by the
critical verb and then by an object NP (and some continuation).
Schmauder's (1991)
experiments used three paradigms: (a) eye movement recording as subjects
read sentences, (b) cross-modal naming, and (c) CMLD. In all three
paradigms, Shapiro et al. 's (1987) verbs were used, with
one extra verb for each verb category added for counterbalancing purposes.
Schmauder created new sentences, similar to Shapiro et al. 's in form, in
which the verbs were embedded. Also, Schmauder used new secondary LD
probes, as described above.
Shapiro et al. (1991) used
a subset of Schmauder's (1991) sentences (those with a
subject NP modified by a relative clause) in a CMLD task. In their
Experiments 1A and 1B, Shapiro et al. used probe items with similar
characteristics to those used by Shapiro et al. (1987)
and found the argument structure complexity effect. In Experiments 2A and
2B, Shapiro et al. used Schmauder's probes, presented either all in
uppercase (Experiment 2A) or in alternating uppercase and lowercase
(Experiment 2B).
Table 1 summarizes the results of the CMLD
experiments presented in Shapiro et al. (1987, 1989, 1991) and Schmauder
(1991).
The experiment reported in this observation
used all of Schmauder's (1991) sentence materials. The
tapes used in the present experiment were the tapes used in Schmauder's
(1991) CMLD experiment, Experiment 4. The secondary LD probe set was
varied as a between-subject variable. One probe set contained Schmauder's
probes. The second probe set contained Shapiro et al. 's
(1987) probes. 1 As was noted in Shapiro et al. (1991), the probes from Shapiro
et al. (1987) were multisyllabic and came from a wider frequency range
than did the single-syllable probes used by Schmauder. Schmauder's LD
targets were 4-6 letters long, with frequency between 50 and 110 words per
million (Francis & Kuçera, 1982). 2
Shapiro et al. 's (1987) LD targets were 5-9 letters
long, with frequency between 56 and 312 words per million (Francis
& Kuçera, 1982). We predicted that if Schmauder's failure to
replicate Shapiro et al. (1987) resulted from using
secondary probes that were less complex than those used by Shapiro et al. ,
we would find an effect of probe set in the present experiment when the
Shapiro et al. probes were used.
Eighty members of the University of
Massachusetts community were given course credit to participate in the
experiment. All were native English speakers and reported normal vision and
hearing.
Subjects were instructed extensively before
the experiment. They were told that sentences would be presented over the
headphones and that at some point during each presentation, a word would
appear on the computer screen in front of them while the sentence continued
to play over the headphones. The task was to decide as quickly and as
accurately as possible whether the target word was an English word or a
nonsense word. Subjects pulled one of two triggers with their index fingers
to respond to the LD target, and they were told to respond as quickly as
possible to the LD target. Particularly slow subjects were informed of
their performance and encouraged to respond faster. Furthermore, subjects
were told that a true-false comprehension question would appear on the
screen after each sentence and that accuracy rather than speed was
important in responding to these questions, which was done by pulling one
of the two triggers. The right trigger was used for yes and true responses,
and the left trigger was used for no and false responses. At breaks between
blocks, feedback was given on percentage correct for LD responses and
comprehension questions.
Details of the experimental design were
identical to those described in Schmauder (1991), the
only difference being the addition of the probe-set variable. Type of LD
target (either Schmauder's, 1991, set or Shapiro
et al. 's, 1987, set) was between-subjects factor. Half of the subjects
saw only monosyllabic LD targets. Half of the subjects saw only
multisyllabic LD targets. Two within-subject factors were used: verb
category and sentence frame. There were five levels of verb category
(transitive, nonalternating dative, alternating dative, two-complement, and
four-complement) and two levels of sentence frame (prepositional phrase and
relative clause). Five pairings of verb to LD target were used. LD targets
did not form plausible completions with the sentential verbs and did not
relate semantically to the sentences. Four randomized orders of
presentation were used, for a total of 20 testing conditions.
The sentence materials and LD probe sets
used are described above. Nonwords for the LD targets were created by
taking words similar to the LD probes for each probe set and altering the
words by changing one letter. The resulting nonwords conformed to rules of
English orthography.
As an initial step in the analysis of
results, data points greater than two standard deviations from a subject's
mean were eliminated. This procedure is equivalent to that used by Shapiro and colleagues (1987, 1989, 1991), in which data points lying more than two standard
deviations from a subject's mean were replaced with that subject's mean
score. This manipulation affected approximately 5% of each set of data.
Note that Schmauder (1991) used reaction time (RT)
cutoffs of 1,000 ms, 200 ms, and three standard deviations. The present
pattern of results is not altered by using these stricter cutoffs.
Mean LD latencies and error rates are
provided in
Table 2. Subjects' accuracy in the LD task
and on the comprehension questions was always above 90%, so we are
confident that subjects followed instructions and comprehended sentences
adequately.
Analyses of variance (ANOVAs) on both
accuracy of LD responses and RTs to LD probes were run with subjects and
with items treated as random variables. We first present and discuss ANOVAs
for the RT data. The ANOVAs of LD RTs showed that the main effect of LD
target group was non-significant, F1(1,78) = 0. 629, p
> . 5, MSe = 108,923, F2(1,50) = 0. 503, p
> . 5, MSe = 2,028. The verb category main effect
also was not significant, F1(4,312) and F2(4,50) < 1 F1
MSe = 4,778, F2 MSe =
2,028). Furthermore, there was no main effect of sentence frame, F1(1,78)
= 0. 284, p < . 60, MSe = 5,634, F2(1,50)
= 0. 9825, p < . 67, MSe = 1,315. The Verb
Category × Sentence Frame interaction was significant by subjects but not
by items, F1(4,312) = 3. 05, p < . 017, MSe
= 5,698, F2(4,50) = 1. 67, p < . 17, MSe
= 1,315. Finally, the LD Target Set × Verb Category × Sentence Frame
interaction failed to reach significance, F1(4,312) < 1, MSe
= 5,698, F2(4,50) = 1. 4, MSe = 1,315. 3()()
Although the only hint of any effect in the
overall analysis was the interaction of verb category with sentence frame,
because Shapiro et al. (1991) found probe-set
differences, we ran separate ANOVAs for the two LD target sets. These
analyses yielded results like those from the overall analysis. In
Experiment Set 1, LD targets were those previously used by Schmauder
(1991), and ANOVAs yielded the following results. The verb category
main effect was nonsignificant, F1(4,156) and F2(4,25)
< 1 F1 MSe = 4,274, F2 MSe
= 909). There was no main effect of sentence frame, F1( 1, 39) and
F2(1,25) < 1 F1 MSe = 5,135, F2
MSe = 1,202). The Verb Category × Sentence Frame
interaction approached significance by subjects but not by items, F1(4,156)
= 2. 35, p < . 056, MSe = 4,594, F2(4,25)
= 1. 82, p < . 16, MSe = 1,202.
In Experiment Set 2, LD targets were those
previously used by Shapiro et al. (1987). The verb
category main effect was nonsignificant, F1(4,156) and F2(4,25)
< 1 F1 MSe = 5,283, F2 MSe
= 3,147). The main effect of sentence frame by subjects also failed to
reach significance, F1(1,39) = 0. 4201, p < . 53, MSe
= 6,131, F2(1,25) = 1. 27, p < . 27, MSe
= 1,428. The Verb Category × Sentence Frame interaction was not
significant, F1(4,156) = 1. 42, p < . 23, MSe
= 6,802, F2(4,25) = 1. 23, p < . 32, MSe
= 1,428.
In the analyses of accuracy data, the only
effect that reached significance was the Sentence Frame × Probe Set
interaction, which was significant by subjects, F1(1,78) = 5. 085,
p < . 03, MSe = 0. 0079, and neared
significance by items, F2(1,50) = 3. 246, p < 0. 07, MSe
= 0. 0018. Subjects seemed to respond more accurately to the Schmauder
(1991) LD probes when they occurred as subjects heard relative clause
sentences (98% accuracy) than when these probe words occurred as subjects
heard prepositional phrase sentences (96% accuracy), whereas subjects
responded more accurately to the Shapiro et al. (1987) LD
probes during prepositional phrase sentences (97% accuracy) than during
relative clause sentences (96% accuracy). We believe that this resulted
from idiosyncratic relations between LD probes and sentence contexts.
In the accuracy data, the Verb Category ×
Sentence Frame interaction approached significance by subjects, F1(4,312)
= 2. 07, p < . 08, MSe = . 0081, but not by
items, F2(4,50) = 1. 46, p > . 2, MSe
= . 0018. This small, nonsignificant tendency reflected slightly higher
accuracy to probes after transitives and two-complements in prepositional
phrase contexts and a lack of such a tendency in relative clause contexts.
Note that for all of these accuracy effects, the range in errors is from 2%
to 5%.
The RT results suggest that regardless of
the type of LD target, verb category did not consistently influence time to
perform the LD task. Furthermore, both types of LD targets showed similar
patterns of results for the two within-subject factors. The only
significant effect found in the three sets of analyses of RT data was the
Verb Category × Sentence Frame interaction, which was found to be nearly
significant in the subjects' analysis for the monosyllabic LD target group
and to be significant for the combined group of both types of LD targets.
The significant interaction appears to be due to fast responses to LD
targets following transitive verbs that were presented in prepositional
phrase sentence contexts.
Calculating the 95% confidence interval
based on the MSe from the subjects' analysis yielded a
value of 23. 4. This indicates that, in the prepositional phrase sentences,
RTs for LD targets following the transitive verbs were significantly faster
than those for targets following nonaltenating dative verbs, two-complement
verbs, and four-complement verbs. For the relative clause sentences, there
were no significant differences between verb categories. Although faster
responses to targets presented after transitives in prepositional phrase
contexts is remotely suggestive of an argument structure complexity effect,
the overall results of this experiment do not support such a conclusion. No
differences appeared between RTs for probes that follow two-complement
verbs and RTs for probes that follow four-complement verbs. No such pattern
was evident in the relative clause materials. Results for the Shapiro
et al. (1991) probes did not reflect argument structure complexity
effects despite the fact that we had ample power in the present experiment
to detect an effect the size of that reported by Shapiro et al. A power
calculation showed that
1 − β was = . 95 to detect an argument
structure complexity effect the size of Shapiro et al. 's (φ = 1. 94).
Clearly, something other than differences
between secondary LD probes alone is responsible for the difference between
the results reported by Shapiro et al. (1991) and those
of the present experiment. We did not find argument structure complexity
effects with probes used by Shapiro et al. (1987, 1989), which had the characteristics suggested by Shapiro et al. (1991) to be necessary for detecting verb
complexity. This was true even though the Shapiro et al.
(1987, 1989) probes were similar on relevant
dimensions to the Shapiro et al. (1991) probe items that
enabled detection of verb complexity.
Furthermore, even if some results are
attributable to characteristics of the secondary probes, the effects did
not generalize to eyetracking (Schmauder, 1991). This
shows that complexity caused by the number of argument structure frames
does not result in a processing cost evident during normal, silent reading,
whereas effects of factors like length, frequency, and syntactic complexity
are evident during silent reading (Henderson & Ferreira,
1990; Rayner, Sereno, Morris, Schmauder, & Clifton,
1989).
From the data reported by Schmauder
(1991) and the results of the present study we can draw some
conclusions. The argument structure frame complexity of verbs that are
being read does not influence eye movements. Argument structure complexity
effects are not consistently detectable with the one task, the CMLD task,
which has been reported to be sensitive to the effect. Shapiro
et al. 's (1991) proposal that secondary probes must be complex enough
to place a significant processing load on subjects for the effect to be
observed does not explain the disparity between Shapiro et al. 's and
Schmauder's results in a straightforward way. Also, although it is possible
that a verb's argument structure is accessed at some point during sentence
processing (see Canseco-Gonzalez, Shapiro, Zurif, and Baker,
1990, for evidence of use of argument structure in an off-line task),
the data reported by Schmauder (1991) and the results of
the present study suggest that at least one of the following may be the
case: (a) that immediate access of argument structure does not always
impact verb processing, (b) that argument structure is not accessed
immediately when a verb is processed, or (c) that the only on-line task
that is sensitive to the effect interacts with characteristics of materials,
instructions, or subjects in some way that has not yet been discovered. In
summary, we feel that our results imply that a more complex argument
structure does not necessarily increase sentence processing difficulty and
that the technique that Shapiro and his colleagues developed for exploring
argument structure has some limitations on its generality that are not yet
understood.
1
We thank L. Shapiro for providing the set of
lexical decision probes used in Shapiro, Zurif, and Grimshaw (1987).
2
Actually, because of an error, one probe was
of frequency 12 words per million, although its verb sense was within the
specified frequency range.
3
Shapiro, Brookins, Gordon, and Nagel (1991) grouped transitive, nonalternating dative, and alternating
dative verbs into one subexperiment and two-complement and four-complement
verbs into another subexperiment for theoretical reasons. As the focus of
the present experiment is on the existence of any consistent differences
among verb categories and not on the theoretical shift articulated by
Shapiro et al. , we analyzed all verb categories together in our primary
analysis. However, we also analyzed our data separately for the two groups
of verb categories. Analyses with subjects and items treated as random
factors showed no effect of probe set, verb category, or sentence frame for
the transitive, nonalternating dative, and alternating dative verbs. Verb
category (transitive, nonalternating dative, alternating dative) interacted
with sentence frame (prepositional phrase, relative clause), F12, 156 = 1.
67, p < . 001, F22, 15 = 4. 4, p < . 03
, because LD responses in prepositional
phrase contexts were fastest for probes following transitive verbs but in
relative clause contexts were slowest following transitive verbs. For the
two-complement and four-complement verbs, all Fs were less than one. In
short, there were no consistent differences among verb categories
attributable to argument structure complexity.
1. Canseco-Gonzalez, E., Shapiro, L. P.,
Zurif, E. B. & Baker, E. (1990). Predicate-argument structure as a link
between linguistic and non-linguistic representations. Brain
and Language, 39, 391-404.
2. Francis, W. N. & Kuçera, H. (1982). Frequency analysis of English usage. Boston, MA:
Houghton-Mifflin.
3. Henderson, J. M. & Ferreira, F.
(1990). Effects of foveal processing difficulty on the
perceptual span in reading: Implications for attention and eye movement
control. Journal of Experimental Psychology: Learning, Memory, and
Cognition, 16, 417-429.
4. Rayner, K., Sereno, S. C., Morris, R. K.,
Schmauder, A. R. & Clifton, C. (1989). Eye movements and
on-line language comprehension processes. Language and Cognitive
Processes, 4, SI 21-49.
5. Schmauder, A. R. (1991). Argument
structure frames: A lexical complexity metric? Journal
of Experimental Psychology: Learning, Memory, and Cognition, 17,
49-65.
6. Shapiro, L. P., Brookins, B., Gordon, B.
& Nagel, N. (1991). Verb effects during sentence processing. Journal of Experimental Psychology: Learning, Memory, and
Cognition, 17, 983-996.
7. Shapiro, L. P. & Levine, B. A.
(1990). Verb processing during sentence comprehension in
aphasia. Brain and Language, 38, 21-47.
8. Shapiro, L. P., Zurif, E. & Grimshaw,
J. (1987). Sentence processing and the mental representation of verbs. Cognition, 27, 219-246.
9. Shapiro, L. P., Zurif, E. & Grimshaw, J. (1989). Verb processing during sentence
comprehension: Contextual impenetrability. Journal of Psycholoinguistic
Research, 18, 223-243.
Received: January 18, 1991. Revised: April
8, 1991. Accepted: April 23, 1991.
|
