Inna Kozlova

Studying Problem Solving through Group Discussion in Chat Rooms

• monolingual dictionary (Collins English Dictionary, 2000) installed on the hard disc of each computer;
• bilingual English-Spanish Spanish-English dictionary (Collins Spanish Dictionary, 2003) installed on the hard disc of some computers (one or two per group);
• access to the Internet in case participants wished to consult online dictionaries or other resources.

• the task in general (1, 2),
• a text or its element (3, 4, 5, 6, 7, 8, 9, 10, 11),
• the error (12, 13, 14),
• a classmate’s suggestion for correction (15, 16).

We could see that in the overwhelming majority of cases the word 'understand' was used to make reference to some comprehension problem concerning the text or its element while its use in other contexts was rather limited. We decided to accept this word as an indicator of comprehension process (labeled as C) though specifying that it had to make reference to the source text or its element to be regarded as such.

All the indicators taken as working hypotheses were analyzed in a similar way  (for details see Kozlova 2005: 307-316). The results of this validation are presented in Table 3. As we can observe from the first part of the Table referring to C and P processes, ‘solution’ was eventually not accepted as an indicator for production, ‘put’ was accepted without any restriction and ‘change’ was accepted with restrictions.

PROCESSES	ACTION INDICATORS
Comprehension (C)	understand (when a reference is made to the source text or its element)
	mean (in any form, such as means, meaning, etc.)
Production (P)	put
	change (when a target text element is present)
PHASES OF INTERNAL SUPPORT
Identification of a problem (I)	problem (when features are absent)
	don’t know (when features are absent)
Definition of a problem (D)	problem (when features are present)
	don’t know (when features are present)
Generation of possible solutions (G)	think (except for when it is a question)
	maybe (except for when it is an answer)
	possible (except for when it is an answer)
	could be (except for when it is an answer)
	can (except for when it is a request)
Evaluation of possible solutions (E)	better
	is correct
	is not correct
ADDITIONAL PHASE OF EXTERNAL SUPPORT
Resources use (R)	dictionary
	Google
	Altavista
	translator

Table 3: Indicators of processes and problem-solving stages after their validation in the entire corpus


In reference to internal support, both problem-identification and problem-definition phases seemed to be marked by the same elements: ‘problem’ and ‘don’t know’. We found that the difference between the two lies in the presence or absence of particular features of the problem. If these are present, like in the example “I don´t know, “in” or “on”, these words are treated as D-indicators, if not, then they are I-indicators.

As for generation of possible solutions, we had to exclude some specific cases: for instance, when ‘think’ was part of a general inquiry “What do you think?” or when ‘possible’ and ‘could be’ were used to support somebody’s suggestion.   
All the 13 occurrences of the word ‘dictionary’ in the corpus appeared to be related to the intention of the participants to look up a certain element. The same happened to other, less frequent, indicators.


3. Data Analysis and Results

The data about the acceptability of results, making reference to the product, was collected after evaluating each group’s corrections. Given that each group had to correct 10 mistakes in the assigned text, the maximum acceptability is 10 out of 10. The aforementioned product analysis was later contrasted with the process analysis.


3.1. Hypothesis 1

To prove this hypothesis, we analyzed the sequences leading to acceptable solutions (A-sequences). First, with both C- and P-indicators present, then only with P-indicators and, finally, only with C. The sequences, coded according to the group and the problem number, are presented in their reduced form in Table 4.

Code1	Sequence2	Result
T7	CCPPP	A
A3	CCCCCCCCCCCPP	A
T9	PCPPP	A
A9	PCP3	A
V2	P	A
V3	P	A
W2	PPP	A
W3	P	A
W5	PP	A
T4	P	A
P5	P	A
A8	P	A
A10	P	A
P9	P	A
A5	P	A
M7	P	A

Table 4: Problem-solving sequences which led to acceptable results


As only the first 2 out of 16 A-sequences (T7 and A3) followed our hypothesized order - production follows comprehension in text reproduction - we have discovered that there were more cases to consider. Namely, the sequences T9 and A9 started with production, then went to comprehension and later returned to production. We could question if this order is efficient as at the end of the day the participants had to follow our hypothesized C->P order. We found a lot of acceptable sequences where only the P-indicator was present. In contrast, there appeared to be no A-sequence at all where only the C- indicator was present. 

We could interpret these results by suggesting that comprehension is the part of text reproduction that is taken for granted by students and is often performed automatically when it presents no problem, which is why it is not verbalized. In contrast, production is perceived as an explicit and final task, so it is always present. In Figure 1 we provide a summary of possibilities for comprehension and production sequencing in text reproduction.



Figure 1: Three variants of comprehension and production sequencing as found in problem-solving with acceptable results


3.2. Hypothesis 2

The next hypothesis was aimed at discovering the relationship between the number of indicators of internal support (I, D, G and E-indicators) and the acceptability of results. We hoped to prove that a more in-depth cognitive implication leads to better results. For this purpose, we analyzed all the problem–solving sequences where any of the aforementioned indicators appeared. Then, we calculated average density for each indicator in each sequence, with both acceptable and unacceptable results. To provide an illustration, we found 92 instances of G-indicator in the total of 33 A-sequences and 72 instances of G-indicator in the total of 27 NA (Not Acceptable)-sequences which gives us G-indicator density of 2.7 for both A- and NA-sequences as presented in Graph 1.



Graph 1: Internal support indicators’ density in A (acceptable) and in NA (not acceptable) solutions


We can observe that, contrary to what we hypothesized, the density of internal support indicators was generally higher in sequences that lead to NA results, which was the case of I, D and E-indicators. This means that the students’ cognitive effort was greater in sequences with NA solutions but nonetheless they failed. As for the density of G-indicator, we found that it was the same in A as in NA sequences, which may indicate that the effort at this stage is not related to the results. This phenomenon could be explained by the fact that students dedicate the same cognitive effort to each sequence and, when at a loss, dedicate more effort to identifying and defining the problem, and normally then there is more controversy within the group concerning the final decision.


3.3. Hypothesis 3

To prove that a more in-depth query of external reference sources leads to better results we analyzed all the R-present sequences (listed in Table 5).

Code	Sequence	Result
V2	GIGRIRGPE	A
P3	(D)IRRRR	A
A1	(G)DGGRRGGI	A
A3	CRCCC(R)CIGGECCIRCRGGCCEGCRGEGEGEIPGP(R)ERGEGE	A
A6	GG(D)(R)GEGGEGE	A
A7	G(R)G	A
A9	(G)E(G)RRRGGGPIGEIGGEG(C)P	A
W7	GPRPGCGDCGGECIPGGPGEGPCGPCCEPEGGEPGPPPGGP	NA
T5	(G)ICCRGEICECIICCCCGI	NA

Table 5: The R-present sequences and their outcomes


The result of our analysis is shown in Graph 2. While the majority of groups used mostly internal support, group A (the group that worked on the text titled Abortion) combined both internal and external support. As a result, while the majority of groups had an average of 5 acceptable solutions out of 10, in group A this number increased to 9.



Graph 2: External support as related to acceptability of results


These results posed the question of why group A was able to achieve such an extraordinary result, as compared to other groups. To answer this question, we analyzed the sequences of group A as contrasted to the total of the other groups’ sequences. The results appear in Graph 3. We can observe that not only are there more R-indicators per sequence in group A, but also there are more G- and E-indicators, which means internal support in group A was more intense than in the rest of the groups as demonstrated in Graph 3.



Graph 3: Internal support as related to the acceptability of results (group A is contrasted to other groups)


This conclusion takes on particular importance when considered in the context of our Hypothesis 2. Previously, we failed to prove there is a relation between more intense internal support and better results. Now, our findings indicate that more intense internal support does help better results, though only when combined with external support.


4. Conclusions and Discussion

4.1 Study design

The design of our study proved itself to be effective and the method of labeling results very useful. However, we should mention some drawbacks to this method. First, it is necessary to remember that this method only allows problematic processes to be recorded while automatic processing relying on competences is not verbalized and is left out of focus. Second, each problem-solving phase allows for innumerable realizations and we had only the most common indicators of problem phases, which were characteristic of our participants.  On the other hand, sometimes no specific word marked the phase but it was presupposed, like in dictionary quotation in A6. We recognize that our system of labeling in its present form could be improved.

In reference to the task design, we should be aware of the fact that students’ interpretation of the task may not completely coincide with our idea of it as researchers. First, student perceived the task as a kind of game, given that communication took place in the same classroom via computer. The task would certainly gain in authenticity if chat conversations were carried out from different locations and asynchronously, which would also allow more time for students to consult external sources. At present, we observed that the students seemed to assign comparatively little value to resources and relied instead on their own knowledge and intuition, an attitude questioned in the present study. The students’ evaluation criteria were their personal satisfaction, which of course often did not coincide with the teacher’s decision on acceptability. We expected our students to consult resources more and use their knowledge to evaluate solutions obtained from dictionaries, thus getting closer to the teacher’s normative perspective. Most students failed to obtain the necessary external support, although they were aware of insufficiency of their own internal support: the students’ global satisfaction with their results (64%) almost coincided with the proportion of solutions qualified as acceptable by the teacher (66%), as the post-task questionnaire revealed.


4.2. Study conclusions

Our study discovered the following tendencies:

• In text reproduction, the comprehension process should be finished before the production process can be finished.
• A more in-depth query of external resources is directly related to better results.
• A more in-depth cognitive implication alone seems not to be related to better results. However, a more in-depth cognitive implication supported by external inquiry is related to better results.

Scripta Manent 3(1)

Contents

» J. Krajka
Online Lexicological Tools in ESP – Towards an Approach to Strategy Training

» A. Curado
Fuentes
Lexical Acquisition
in ESP via Corpus
Tools: Two Case Studies

» I. Kozlova
Studying Pproblem Solving through
Group Discussion
 in Chat Rooms

» M. Šetinc
Militarwörterbuch Slowenisch-Deutsch (Wehrrecht und Innerer Dienst) / Vojaški slovar, Slovensko-Nemški (Vojaško pravo in Notranja služba) and
Militarwörterbuch Slowenisch-Deutsch (Infanterie) / Vojaški slovar, Slovensko- Nemški (Pehota)
A review

Other Volumes

» Volume 2/2

» Volume 2/1