For a programme to be successful these elements must be coherent and, crucially, the online element must not be seen as an extra but must be central to the process of learning and of assessment.

In this section the impact on learning of each of the sections is considered in turn.

4.2 The impact of online work on learning

The majority of online learners report that they find one or more of the online aspects useful (see Table 4.1 below). Frustrations occur when material is not available for them to contribute to, or when their contributions go unanswered. These frustrations are often voiced in a need for more face-to-face meetings (see the end of this section), a natural reaction but one which needs to be first addressed by ensuring that discussions are available, interesting and relevant.

59 cohort 1 NPQH learners were surveyed in July 2001 and asked which of the elements they found most useful to their learning.

The results showed that they found the national overarching elements of most use.

Online element	Hotseats	Website of materials	Regional discussions	Learning space
Number of mentions	26	21	6	13

Table 4.1 Responses to 'Which element/s do you find most useful?' July 2001

The most often stated reasons given to explain the usefulness of these elements to learning were:

Hotseats and the website of materials are the two most popular elements. It is these two elements that allow for the more reflective style of learning with learners being able to spend time reading, rather than contributing.

To balance this it is necessary to take a more proactive approach in the learning spaces and discussion forums where teacher and learner are engaged in discussion. The resulting restructuring of summary of learning groups, following the survey and feedback from centres, has allowed more sharing of learning.

4.3 Logging in rates

For the induction to Cohort 1 in February/March 2001, there was no requirement for learners to log in. Indeed, few induction tutorials were held in venues where this was possible. This caused problems with learners subsequently not being able to access the online components.

As a result of this lack of hands-on access, there were a significant number (1 in 14) of learners who had not logged in after several weeks. After the experience of this cohort, centre managers revised their approach to the induction tutorials so that by Cohort 3, the requirement to log in and experience the online environment was central to the tutorial. Following the introduction of integrated ICT in the recommended agenda for induction tutorials (cohort 3) the log in rate for all learners had increased to a level at which virtually every learner had logged.

Table 4.2 Log on rates

4.4 Summary of Learning Areas

In Cohort 1, the summary of learning community was shared between one tutor and one learner. This required the tutor to set up and manage a large number of communities, maybe as many as 30 if they work on both Access and Development stage. Across the country this resulted in some 2000 communities, with a consequent problem of navigation and location of previously-visited conversations. In some cases, where participants provided impressive Summaries of Learning these were locked in 2 member community isolation, unshared and uncelebrated. This was completely contrary to the philosophy of both community and the programme, which emphasised collaboration and shared learning.

Image 4.1: A Cohort 1 summary of learning, showing little interaction

Often learners did not know how to post to their summary of learning space, as this had not been covered in the induction tutorial. With so many communities to visit, successful posts were often not seen by tutors for some time.

This structural issue was addressed for Cohort 2, by combining the summary of learning spaces of all learners that shared a tutor to form a 'Shared Learning Space'.

Image 4.2, Example of a shared learning space, increasing interaction via a shared single page

As well as making the navigation for tutors easier, the reduction in the number of Summary of Learning Communities (from approximately 2000 to 250) meant that they could be set up centrally in each region. This meant fewer errors and less problems for learners in finding their spaces. This had been a major source of frustration for cohort 1 learners:

        "The site is hard to navigate"

        "I have too many communities"

Versions 2 and 3 of think.com and the simplified structure of communities were both welcomed, and are having a positive impact on online learners' use of these spaces. In Cohort 3, four Summary of Learning Spaces were analysed. Over 75% of candidates make regular use of their summary of learning, posting a summary at least once a month. In cohort 1 the figure was under 50%.

Learners' responses often reflect the attitude of their teacher to ICT and online teaching.

4.5 Hotseats

From the first hotseat run by Michael Barber the hotseat programme in NPQH has been hugely successful and there is ample evidence that online learners in NPQH are using the archive to further their learning:

Image 4.3, Hotseat page with Tom Brighouse

Conversely learners have high expectations of the hotseats and become frustrated when the discussions are not easy to follow:

One of the aims of the review of the NPQH programme model was to provide interaction with serving headteachers. This is appreciated by learners:

        "Ones with real heads are best... "

        "The best hotseat was the one with the teaching head... "

        "The head of the special school raised my awareness of SEN issues... "

Another reason cited for their popularity is their involvement of a large number of fellow learners:

       "They allow me to get a wide range of different viewpoints and perspectives."

       "It is good to share with existing heads and fellow candidates."

Conversely though, some candidates feel this constrains their use of this forum:

       "The hotseats are interesting to read, but I feel vulnerable in a big group."

       (Learners' feedback, July 2001-June 2002)

The archive provides a dynamic supplement to the materials website. In compiling the archive an index has been created that collates the hotseat summaries according to the module that they address. Links between the programme materials and hotseat discussions are being explored in the rewriting of the materials.

Image 4.4, hotseat archive

The hotseats were initially envisaged to cover two aspects of learning:

 In parallel to this model, the questions and answers were initially of the 'learner-expert' type, with learners asking direct questions and not reflecting on their practice. As the hotseat schedule proceeded the questioning became more reflective.

Several hotseats have been analysed for this change of style. Evidence of learning based on learners moving from stating what they do now, through reflection on others' comments to direct expression of a change in practice. The figures for this last type of question are suppressed as few learners ever return to a hotseat to report back such change of practice.

4.6 Analysis of learner contributions to hotseats.

Four categories for the contribution of learners were recorded. These are subjective categories, the criteria determining each is shown in Table 4.3 below. It should be noted that it's common to find contributions that fit into more than one category.

Hotseat guest as expert; case study	Mainly Q & A - talking about the hotseat guest's own school, their expertise.
Me and my school	The contribution is characterised by the learners repeatedly referring to themselves, their school or their colleagues.
Reflection of what has gone before (learning)	Reference to other questions/answers. Mention of others, explicit references to learning.
Impact, evidence through actions	"As a result of this I will/have... " The hotseat has caused change to happen.

Table 4.3: the four categories of contribution from learners in hotseats.

Exemplars were collected for three of the categories, the exception being 'Hotseat guest as expert'.

The figures that relate to the categories are in Table 4.4.

Hotseat Guest	BM	DF	PS	GG	TB	TOTAL
Hotseat Guest as expert/case study	10	9	6	1	18	44
Me and my School	13	25	7	7	10	62
Reflection on previous comments (learning)	14	21	9	7	19	70
IImpact, evidence through actions	1	0	0	2	1	4
TOTAL	32	51	22	12	46	180

Table 4.4: Categorisation of learners' contributions to hotseats.

Clearly learners are not reporting impact in the hotseats. This may again reflect their national status and their phrasing in question and answer with a guest. The impact on actions is a specific requirement of the school-based activities, which are reported on in candidates' summaries of learning. It is interesting to note, however, that learners are interacting with each other, reflecting on each others' comments and on their situation and their schools, with over 70% of contributions falling into these two categories.

Examples of comments showing impact:

4.7 Materials website

Each module emphasised reflection and 'cued' online learners for the reflective experience. The materials website consistently linked back to the online community and gave online learners the opportunity to develop a pedagogy based on dialogue, discussion, collaboration and sharing best practice within the communities to which they belonged.

Thus the materials website was consistent with the changed philosophy of the NPQH programme; this was not a managed learning environment in which control resided in the programme team, but an information tool through which the pedagogical thinking residing in the programme (self-directed learning, continual professional development, collaboration, sharing best practice and criticality) could be best implemented.

Image 4.5, materials website

The NPQH learners had access to the materials in paper form as well as on the website. Both of these formats are valued.

        "The paper materials are wonderful... "

        "The website materials and their links are very convenient... "

            (Feedback from learners, July 2001)

From the launch of the new model of NPQH the website of materials has attracted a large number of visits with each visitor spending some time browsing and reading a number of pages.

Duration of logging: 8 months

Number of page hits: 119,000 (exactly)

Number of sessions *: 10198

Number of pages per session: 11.7

* a session is defined as a visit by a user from logging into the site, to

leaving it.

4.8 Module discussion groups

Known as online tutorial groups in Cohort 1, these provide a space for learners across a region to share in discussion with the regional tutor team.

(Centre manager comments on Cohort 1)

Image 4.6, module discussion group

4.9 Centrality of online elements to learning and assessment

From the outset, online work was seen by a 'core' of online learners as an integral part of the programme. Many of the early complaints concerned the inability of learners to find appropriate online spaces, either due to the failure of online teachers to establish these areas or navigation complexities.

Even in early discussions there was obviously some reflecting of ideas back into the school situation. Cross-LEA boundaries aided this discussion. Online learners report they have benefited from access to a much wider range of ideas online, for example hotseats allow them to directly interact with experienced headteachers and policy makers and to share ideas.

In any programme of learning, components need to be coherent, and seen by learners to contribute to the whole programme, and to its assessment. This applies equally to the online aspects of the NPQH learning programme. Without this centrality, learners will become frustrated and put undue reliance on the face-to-face elements of the programme.

These issues are compounded if, and when, learners realise that their online contributions do not directly form part of the evidence for their assessment. To counter this, many regional centres are asking assessors to explicitly require learners to produce the evidence so that it may be considered along with that from the school-based activities.

4.10 Impact of online learning on learner's ICT capabilities

While the emphasis of any online component of a learning programme must be on the core topics being studied e.g. leadership, management, learners inevitably report a benefit in terms of their capability with ICT and the flexibility it brings to learning.

Indeed, it was from the evaluation of the pre-2001 NPQH programme, indicating the desire of learners to use ICT in their learning, that led to the introduction of the online elements in the new model. In the first cohort, 28% of candidates registered online. This increased to 55% in the second cohort, as the centrality of ICT in the process became recognised.

Back to contents

Section 5