The Sutherland Inquiry into National Curriculum Testing in 2008

The full report can be accessed here.

The responsibilities for failure are made crystal clear.

Page 1
“The primary responsibility must therefore rest with the American organisation, ETS Global BV (ETS), which won the public contract to deliver the tests and failed its customers.”

“The events of this summer also represent a failure on the part of one of the Government’s Non-Departmental Public Bodies, the Qualifications and Curriculum Authority (QCA), to deliver its remit. ... In practice, the first time QCA notified Ministers that ETS would not deliver test results on time was 30 June 2008. The whole episode was punctuated by similar instances of poor communications, whether to schools, to the marking community, or between the organisations involved.”

Nowhere is it suggested in the report that the number of links in the chain of command makes the effective completion of the marking of tests more difficult. I find it revealing where Jim Knight comments on what happened in July 2008
“That is when the whole business of the relationship between us and QCA, NAA and then the contractor started to come sharply into focus, because clearly we were asking for things from QCA that we then were not getting. You then had to decide whether or not to forgive the QCA because it was actually the contractor’s fault, or whether it was a problem with the QCA. In the end, I had to take the view that our relationship was with QCA, it wasn’t with the contractor. It was up to QCA to come up with the answers and solutions to the problems. How they did it was up to them, they just had to deliver. And frankly, they weren’t.” (Page 80)
In other words DCSF – QCA – ETS chain meant that the government was dealing with the developing crisis at one remove from the operational end.

The inquiry report doesn’t consider the whole contracting approach to tests as a possible source of difficulty. Private sector commissioning of public sector activity is so habituated within government strategy that the question doesn’t even seem to have been asked. Rather Sutherland focuses on the detailed arrangements between all these parties assuming that the solution to the problem lies within them.

Page 35 “The Secretary of State the Rt Hon Ed Balls MP has described:
‘…in terms of the delivery of tests, I don’t think there was ever any doubt that there should be an agency function. It is not sensible for ministers to get involved in the details of the procurement process, but actually the kind of expertise that you need in order to manage a complex contract, isn’t the same as the kind of expertise you need to make good policy…’”

So the advisability of external commissioning of testing is not questioned, only the separation of strategic and operational functions.

Page 36
“DCSF Permanent Secretary David Bell has described the department and QCA’s respective roles:
‘…what we do is delegate by remit letter essentially a large number of our delivery responsibilities. And to some extent that reflects the theology of the last twenty years or so as central government departments have increasingly passed over responsibility to other government agencies…’”
What an interesting choice of words, ‘theology’.

Page 37
“QCA Chief Executive Dr Ken Boston has stated that:
‘Government is at arm’s length only from the detail of the test questions and from the marking and level-setting… Throughout the process of procuring the contract and delivering the tests according to DCSF specification, ministers and officials had access to exactly the same data and information as the NAA and the QCA; they were active participants in the process; and they provided a separate source of advice to ministers. They were properly part of the process, and in no way at ‘arm’s length’.’”
If this is the case it makes Ed Balls comments from page 35 seem misguided.

Questioning the use of private sector providers for public sector functions as vital as national testing, seems unlikely, given David Bell’s view of the current ‘theology’. If private sector involvement had been within the scope of the Sutherland Inquiry and present difficulties had been contrasted with performance when these functions were delivered by the public sector, the conclusions might have been very different.

Interim Report on the Primary Curriculum

This has just been released and can be downloaded from here.

The press attention has been around the recommendation that primary schools change their organisation from a subject to a more thematic approach. This reflects much of what has been going on in the secondary curriculum of late and is very relevant to the House Committee on Children, Schools And Families examinations that were reported in my last post.

Sir Jim Rose acknowledges the tensions between the two approaches to organising learning and states; "six areas of learning are proposed to give schools optimum flexibility for planning cross- curricular studies, and ample opportunities to teach essential content discretely and directly". In other words lets give schools the ability to do some thematic work but at the same time design the curriculum so that they are influenced to teach the really important building blocks of content. A recommendation is..

"Given the excellent examples of both witnessed by the Review, neither discrete subject teaching nor cross-curricular studies must disappear from primary schools. Schools should protect time when learning is best served by teaching subject content discretely and systematically, and give children ample opportunities to use and apply their developing subject knowledge, skills and understanding in cross-curricular studies."

I am pleased that he highlights the need to develop young peoples' ability to speak and listen.
"Discussion of reading, writing and numeracy in primary education often fails to recognise the central importance of developing children’s spoken communication."

ICT is also given a strong role in helping improve the learning that goes on in primaries. He states "One highly promising route to meeting the demand for in-depth teaching and learning is undoubtedly emerging through ICT." It is clear that he believes that ICT capabilities need to be explicitly addressed and also that ICT itself is an enabler of enhanced learning across all subjects (and themes).

I await the final report with keen anticipation

House Committee on Children, Schools And Families

Many will have already seen reports of the evidence given to the Children, Schools And Families Committee of the House on the 17th November. Teresa Bergin, Professor David Hargreaves, Tim Oates And Mick Waters were all examined. The reporting has tended to focus on the views expressed on personalisation. These are interesting. For example David Hargreaves was asked about how he defines the term and responded;

'The current thing from the Department quotes the definition given in the Gilbert report on teaching well in 2020, of which I was a member: "It means strengthening the link between learning and teaching by engaging pupils-and their parents-as partners in learning." In my view, that is well-intentioned waffle. It is well intentioned, but it means nothing. In fact, many schools will say that that is what they do. There is no implication of action at all. '

This is unsurprising. In my view personalisation in education policy was an example of strategy flowing from marketing. Number 10 liked the word and it's associations and it was up to the civil servants, education quangos and schools to make something useful out of it. I thought David Miliband's definition at the the North of England Education Conference in January 2004 was the best of many attempts to make the idea coherent. But in the end the poverty of philosophy behind the idea overwhelmed it.
Much more interesting in my view is the discussion of thematic structures for learning.

Hargreaves;
"I know of no hard evidence that teaching thematically, as opposed to under subject labels, would produce better learning, although it might be attractive and engaging to some learners. On the other hand, there is hard evidence that project-based learning-I encourage you to make the distinction-can improve not only students' engagement, but their achievement"

Tim Oates;
"the work of groups such as the science education group at Leeds suggests that unique methodology and bodies of knowledge are intrinsic to subjects. If those are not taught systematically or covered, there will be problems in progression through the education system at all stages. "

So the witnesses described the tensions between project based and subject based learning. A number of very important issues were revealed. Without enormous care project based learning can allow some young people to miss key building blocks in particular subjects. Maths was highlighted as a subject where the understanding and knowledge is hierarchical - a child without a foundational element will never be able to grasp higher concepts.
The issue of rigour was also raised. The witnesses seemed to believe that lack of rigour is a weakness particular to project based approaches. I don't accept this.
I do agree with these very well informed witnesses that the quality assurance issue is key now that central government is beginning to step back from a highly directed approach.

Hargreaves;
"What has been happening in more recent times, and will continue to happen with the loosening up of the national curriculum, is that schools are quite properly trying out new ways in which to engage young people. In my judgment, the centre should be asking, "How can we identify what is really rigorous in what you are trying?" and feeding it back. Matters should be iterative between the centre and the schools. At the moment, we do not have that kind of partnership-we have schools sitting, expecting the centre to dictate to them which does not work. That is the whole point of personalisation. Customisation has to respond to the needs of the clients, but the role of the centre is to say, "Are you sure that you are doing it well?" Although I may be wrong, I am willing to bet that if you asked Ofsted what criteria it uses to judge whether project-based learning is good in schools, it does not have a research-based checklist by which to judge what they observe."

The question of spreading good practice arose from this discussion and it seemed generally accepted by the witnesses that there isn't any one education body with a clear remit for this task. The sharing of good practice by teachers themselves is something I think is much more complex than is generally understood. At one point Mr Chaytor (a committee member) remarks;

"I am amazed that the spirit of Wikipedia has not permeated the whole of the teaching profession, and that teachers in schools up and down the country are not posting up their best ideas."

Actually there are a very large number of websites where teachers share materials. The reason these sites are less useful than they might be is that contexts vary dramatically. One teachers excellent resources are useless for another in a very different school. Teachers own teaching styles vary as well so that resources that work very well for one teacher don't feel right for another. I think teachers rating each others resources is probably the best way to solve this problem. We don't need any more resource sharing websites but something like digg for teachers where they rate resources they find all over the web.

Art History

It isn't often that I come across an online learning resource that really impresses but http://smarthistory.org does. It's a lovely collection of discussions around some key pieces of art from history. The scope is huge covering classical art to modern. In total the website has 157 artworks and a slightly smaller number of podcasts where the works are discussed. These discussions are what I think teachers will find most useful. Although the language is often sophisticated, the two editors Beth Harris and Steven Zucker do make an effort to explain terms like 'neo-platonic' for example. The great strength of these podcasts is that teachers will be able to play them again and again and offer their own support and commentary alongside.

Becta Review 2008

Another report from Becta worthy of consideration by all those who work with schools on ICT development. Accessible here
The figures on teachers attitudes and skills with ICT on the face of it look very good.
77% of teachers either very confident or quite confident in using ICT to deliver the school curriculum. Most teachers enthusiastic in 51% of schools.
Bear in mind that the first of these figures represents the views of an ICT Coordinators. It worries me that only a minority of teachers seem enthusiastic about ICT in 49% of schools. In the work we do with schools teacher attitude is as important if not more important than their skills in determining the outcome of an ICT related project.
The research also asked teachers how effective they thought they were in using ICT to support teaching and learning in the classroom. 77% said they were either quite or very effective. Interestingly there was a strong correlation between years of experience and effectiveness. This result somewhat undermines a common assumption that it is the newer members of the teaching profession who are best at using ICT in schools.

Becta Report on Web 2.0 in Schools

Lots of reports actually, all visible here
http://partners.becta.org.uk/index.php?section=rh&&catcode=_re_rp_02&rid=14543

The summary document contains the following sobering finding for all those Web 2.0 utopians:
"Overall, although most learners use the internet for learning, there is only limited use of Web 2.0, and only a few embryonic signs of criticality, self-management and meta-cognitive reflection."
"Relatively few learners are engaging in more sophisticated Web 2.0 activities such as producing and publishing their own content for wider consumption."
In other words most of our school children are passive users of social websites and Web 2.0 sites in general. The teacher has a significant role:
"Lack of significant sophisticated activity by learners that involves more than consumption and social networking suggests that there is a role for teachers in supporting effective learning using Web 2.0. This role may be to ensure that learners have the technical skills to use the tools effectively and the metacognitive, synthesis and critical reflection skills to use Web 2.0 applications to support learning wherever they are."
I'd suggest these are probably fairly fundamental skills for lifelong learners in most media, traditional and new.
The following is perceptive and matches with my experiences in this area:
"...perhaps the greater challenge is that, at present, school students do not often create – they too often copy and learn. Often, teachers are unable to easily engage in formative assessment procedures with their students. Traditionally, they do not mix media – the standard output from school work remains paper-based. Traditionally, it has been difficult to blur boundaries between school work and homework. Traditionally, authority has had to appear too singularly invested in the teacher or the textbook.
Web 2.0 approaches seem to challenge each of these structures, and replace them with open-ended learning environments and assessment procedures, with mixed-media outcomes that are created and evaluated in new authority and ownership structures. It is hardly surprising if teachers are only exploring these spaces tentatively and cautiously. "

Sanctions and Rewards

The NAO has recently published research examining the use of rewards and sanctions in government. (http://tinyurl.com/4wkxt4) The document considers how effective these are in driving improvements in public sector delivery. Education is mentioned. The report is not impressed by the use of threshold based sanctions such as the present 30% used with schools.
'“threshold” schemes, which target absolute levels of performance and do not take past performance into account, may not reward Agents who improve the most as result of starting from a lower base.' (Page 27)
The material on unintended consequences of reward and sanction regimes should be required reading for anyone working within the public sector.
Of course the question does arise - if the NAO are saying that threshold schemes aren't a good idea - why isn't the government listening. (As if I didn't know).

Becta and Microsoft Kiss and Make Up - Not Quite

As yet limited reaction in the section of the blogosphere I roam through, to this announcement from Becta http://news.becta.org.uk/display.cfm?resID=37559&page=1658&catID=1633 in relation to the long running disagreement between Becta and Microsoft. Just be clear that there is still a complaint from Becta to the EC about Microsoft and this latest announcement hasn't made that go away.

The ongoing bustup was over two issues. First licensing. Now Becta say MS is going to loosen the very tight threads of Schools Agreement Licensing, but in six months... oh and as a pilot.

Microsoft's own Schools Blog http://blogs.msdn.com/ukschools/archive/2008/09/17/becta-pouring-oil-on-troubled-water.aspx reassuringly states "The Becta release is very specifically comparing the new scheme to Schools Agreement, even though the majority of schools don’t use it. So don’t panic – after reading their news release, you might think you’re being forced to license computers you don’t run our software on! You’re not." Phew, what a relief! This blog says nothing about the other dimension of the dispute about ODF document format.

Meanwhile the more balanced Merlin John http://213.232.94.135/merlinjohnonline/news.php?extend.366 (of TES Online fame -God Rest Its Soul) reports "If you were to deduce ... that licence payers were spending more than they needed, that would not necessarily be correct. Licences are often catch-all to give users a free hand across a range of possibilities. They are better value for some than others. And the licensing pilot that Microsoft is undertaking will check out different possibilities to increase flexibility for schools."

So not much of a change then. The other dimension to the dispute is around MS Office 2007 and the ODF format. First MS didn't have Office 2007 open or create ODF documents. Now MS are saying that this will be reversed. ZDNet http://news.zdnet.co.uk/software/0,1000000121,39488825,00.htm quotes Mark Taylor, chief executive of Sirius Corporation and founder of the Open Source Consortium "[Sirius's view] is that Microsoft has been forced to this position, and that the term 'clear commitment' should be read 'dragged kicking and screaming'. If not for the stance of Neelie Kroes and the European Commission, if not for the OOXML roadshow and the ISO controversy, if not for Becta's OFT complaint, does anyone believe this would happen?"

But Becta welcomes the limited moves Microsoft are making.

I just wish Becta would press a well known MIS supplier as hard...

SWE London Research

Full title is "The Interactive Whiteboards, Pedagogy and Pupil Performance Evaluation: An Evaluation of the Schools Whiteboard Expansion (SWE) Project: London Challenge" from the Institute of Education, University of London. This came out last year and not surprisingly I missed it.
There's some very useful and tantalising pointers in the research.
These stand out

"Discussion of pedagogy should precede and embed discussion of the technology. Successful CPD is most likely to be effective if it supports individual teachers exploration of their current pedagogy, and helps identify how IWB use can support, extend or transform this. Discussion of the relative strengths and weaknesses of different ways of using the technology for particular purposes should be part of the on-going work of a department." [page 4]

"When creating their own texts, many teachers struggle to incorporate principles of design which can establish clear reading paths for pupils. Lack of familiarity with such principles of design may make it much harder for teachers to create and share resources that can be used independently of their author." [page 5] There isn't, as far as I can see, any reference to these 'principles of design' and how you'd find them in the main body of the research.

"The literature suggests a continuum in which new technologies initially support, then extend and finally transform pedagogy as teachers gradually find out what the technology can do. Familiarity, confidence and time are assumed to be the keys that unlock this gradual process of transformation." [page 6] AND "the introduction of an IWB does not in and of itself transform existing pedagogies." [page 6]

"The use of an IWB does not of itself automatically alter the dynamic of whole class teaching in secondary core subject areas. It does offer up an opportunity to think about the strengths and weaknesses of whole class teaching and how else it might be organised. Where we observed best practice, departments or individual teachers were aware of this dimension and had consciously set aside time to reflect on the most appropriate use of the technology in their own context." [page 7]

"More open-ended discussion between colleagues needs to take place about how IWBs can be used to support, extend, and transform existing practice. Each of these uses has a value under the right conditions. Teachers should be encouraged to consider when it is appropriate to use the technology for any of these purposes and which aspect of the technology might be most appropriate to achieve that aim." [page 8]

"the real value of IWBs for teaching and learning in different subject areas of the secondary curriculum is not yet fully understood" [page9]

Becta Report on Managed Services

These are a big part of what is being leveraged into schools through the Building Schools for the Future programme. Schools get nice shiny new buildings and in return they pretty much have little alternative but to accept a managed ICT service for their school. So since BSF was first announced in 2003 (or earlier possibly...) it's a bit surprising that in March 2008 Becta publish research addressing how much benefit schools can realise through procurement of a managed service. (now available here)


Since it is a key part of Government policy within the one of the most expensive capital projects of the administration forgive me for being a little sceptical about the findings of this research. The research team certainly included reputable academics but also Becta representatives. The researchers contacted 400 sites running managed services collected online survey data from 79 and then 29 sites were identified to be visited. Inclusion of some sites that had considered and then rejected outsourcing or some that had outsourced and then moved back to in-house provision would have added some balance. The description of the methodology in the report does not indicate if these approaches were even considered. The final list of sites visited is as below

Site, Visits/interviews, Case studies
Primary 5 2
Special 0 0
Secondary 14 4
Academy 1 1
College 4 2
Local authority 4 3
Adult community learning 1 1
BSF 0 0
Totals
Target 30 13
Actual 29 13


(Page 16)

Secondary schools made up less than half the sites visited and they were less than a third of the case studies. It is also interesting that the research team note "In discussion with Partnerships for Schools (PfS) and Becta it was agreed that it is too early to draw any conclusions from the experiences of BSF schools beyond that published elsewhere as part of broader BSF evaluation." (Page 17) and so BSF schools were not included. This is odd given that this research will almost certainly be used to support the use of managed services within BSF and given the immaturity of the outsourced services that were examined. (See below)

The picture the research paints is of outsourcing commonly occuring because the in-house provision is poor and some event highlights the need for change. "A common picture emerged in all site visits of underinvestment in ICT prior to the introduction of the managed service. This was often compounded by the evolution of technical support roles and ICT management processes to a point where they were unclear or ill-defined. In some cases, school leaders did not fully appreciate the development needs of technical staff. There was also a tacit acknowledgment that technicians were not effectively managed with too much autonomy delegated without appropriate accountability. In many cases, a vision for the contribution of ICT to the overall mission of the school and the translation of that vision into practice was either missing or ineffective." (Page 19) To summarise the problem is too little money and poor management. What is unclear to me is how a managed service would change these underlying problems. For example how would a managed service enable a school to articulate a vision, or have more money?

I am also concerned about the immaturity of the relationships described in the report. 20 of the 42 services that the research examined were no more than 1 year old. (Page 27)

Headline findings that "the managed service does deliver improved value for money from that investment" (Page 5), dissolve into less impressive findings in the body of the report "The vast majority of sites in general felt unable to give meaningful comparative figures as the costs associated with ICT were so poorly understood before the managed service. Therefore, the evidence presented here is largely anecdotal and the conclusions that can be drawn are limited." (Page 27) Given also that the research states "Establishments considering employing a managed service need to ensure that a senior member of staff is given sufficient time and resource to oversee and champion the process ideally from procurement through to the final implementation." (Page 7) there are clearly a range of costs that might be considered within the question of 'value for money'. Sadly the report gives very little indication as to how the very tentative conclusions about this were calculated. As to headline spending "Of the 29 sites visited only 14 were willing and able to even approximate their change in financial expenditure linked to the managed service, with nine stating an increase and five no change or decrease. " (Page 108) Surely the key question here is whether this increased spending (on balance) resulted in a better service than increasing spending on the in-house support function? Making things better by spending more money on them isn't the kind of finding you'd want if you were driving through very large scale systemic change.

On page 110 the report makes the following strange claim in relation to the above "Most sites indicated a rise in expenditure. However, further (often anecdotal) exploration indicates that the majority of that spend arises from new hardware procurement and installation and is therefore not intrinsic to the managed service." Anyone with the slightest experience of ICT systems knows how much ongoing costs can be reduced by the procurement of all new and consistently built hardware. This comment along with many others in the report makes me very dubious about recommending its conclusions on outsourcing to schools I work with.

ICT Self Review

There's a very complex and interesting debate to be had about the role students take within state education in the UK. Are they partners, consumers, commissioning agents, subjects or employees? Probably the one thing most educationalists would agree is they are definitely not employees. Are their parents the customer? Are the local employers the customer as they will be consuming much of the finished product?


In terms of IT services, perhaps there might be a case for considering school students to be employees but I’m not convinced. They don’t, perhaps shouldn’t is a better way of putting it, negotiate with IT about the services they require. The school hierarchy does that in their interests. They are, after all, for most of their compulsory education, children, minors and therefore not fully capable of determining what is in their own best interests. I am completely convinced that their views must be listened to, don't think otherwise. But one of the things schools need to do whether they have a managed IT service or not, is to determine what IT services the school wants and what priority each should have.


This determination is close to the heart of the self review process and so it's very important. As I'm starting to work with three schools on the SRF journey these thoughts arose about what the involvement of students might need to be.

SIF

Schools Interoperability Framework - Becta have recently published a of proof of concept pilot that was run in Birmingham. (http://industry.becta.org.uk/display.cfm?resID=28188) This got much much less attention than even I anticipated. After all school information data structures are not exciting, probably not even to those working on them.

So why is this worth anyone's time and interest?

Capita and the SIMS monopoly. The company have had a very easy time of it - hauling in school finances for their product and not much competition out there. They even charge third party suppliers of products that leverage SIMS to access a schools data.

A couple of years ago Becta fired a salvo across their bows with the catchily titled "Research report: School management information systems and value for money A review with recommendations for addressing the suboptimal features of the current arrangements." (http://publications.becta.org.uk/display.cfm?resID=25917) The criticism of Capita therein was surprisingly hard hitting for a quango. And yet not much has happened subsequently.

The SIF if adopted may unlock the potential of the schools data management market. Becta want it as the standard for all MIS systems for schools. The standard will allow software producers to create add-ons to any MIS without proprietary lockins or issues with interoperability. Instead of a SIMS competitor needing to come to market with a fully comprehensive, mature MIS for schools they could develop one incrementally. Replace one 'suboptimal' element of SIMS with their own every so often until all that's left of SIMS is a stub. Schools could mix and match from a number of suppliers. The potential is enormous as up to now, in my humble opinion, schools data management systems have been lacklustre and unhelpful in the task of automating administrative and other tasks for those working in schools.

Interesting to see what happens...

Hello

When starting a blog what is the protocol, appropriate form?

The intention is that this will be the place where I offer my point of view on stuff to do with technology in UK schools. What would anyone get from this. Honestly probably not that much. I read a good deal of material and lead a City Learning Centre that has does some pretty groovy things with technology in schools. But the credit for that goes to my staff. I pick up some good ideas from them.

Mind you the stupidity of ideas contained herein is entirely my responsibility. Finally the thoughts, ideas and opinions in this blog are personal and do not represent any of the organisations I work with or for. Phew...!