The Care Act 2014 outlines the responsibilities of local authorities (LAs) to provide outcomes focused services and encourages them to use national, as well as local, sources of service-user feedback, such as the Adult Social Care Outcomes Framework (ASCOF), to guide the design and delivery of local services.

The data from the adult social care survey (ASCS) and carers survey (PSS SACE) currently populates 13 of the 21 ASCOF indicators and therefore are potentially an invaluable source of data.  Our earlier research activities with LA staff, however, indicate that many organisations limited their analysis of their datasets to descriptives (e.g. frequencies and percentages of individual responses to survey questions) and direct ASCOF comparisons and saw the ASCS and PSS SACE as “performance surveys rather than potential shapers of policy and practice”.[1]

LA ANALYSTS MAY EXPERIENCE A NUMBER OF ANALYSIS BARRIERS

The LA analysts we spoke to described a range of barriers to analysing their ASCS and PSS SACE datasets. These barriers were surprisingly widespread and demonstrated that their approach to describing all the data was logical under the circumstances.
Analysts frequently felt unsure about how to navigate the volumes of data produced by the surveys and struggled to determine which variables to investigate and in what order.  As a consequence, most resorted to exploring their datasets sequentially (in other words, question by question).

Many analysts also required new analysis skills and/or access to additional resources (in particular, statistical software such as SPSS) to conduct more in-depth analysis and were therefore limited to summarising their datasets using descriptives.

A WASTE OF TIME AND A MISSED OPPORTUNITY?

Our discussions with potential consumers of ASCS and PSS SACE data – in other words, managers and commissioner who can use the data to inform their decision-making and service improvement activities – indicated that this approach was not always useful:
They’ve literally gone through each question and just kind of reported on it, but, by the time it goes through to the commissioners, it doesn’t work.  It’s kind of 50 pages of graphs and pie charts, and it doesn’t work
[Commissioner – Source: MAX telephone interview]

As analysts are under pressure to manage increasing workloads, this approach to analysing ASCS and PSS SACE data may not make the most of their precious time. Furthermore, as both surveys produce robust data from such large samples of adult service-users and carers – and is potentially the most significant piece of local research conducted by a LA – not using the data to guide local service delivery and improvements may be a missed opportunity and a considerable waste of resources.
So, what is the solution?

INTRODUCING THE ANALYSIS ELEMENT OF THE MAX TOOLKIT

In-depth statistical and thematic analysis is required to make sense of the outcomes data collected by the ASCS and PSS SACE, and to produce meaningful information that can be used to guide local decision-making and service improvements. Support, however, is clearly needed; In fact, support to analyse and interpret survey data were the most common requests during our earlier MAX online survey (see Figure 1 below)

We developed the analysis element of the MAX toolkit to provide this support and have included guides, excel-based tools and training resources to help LA analysts to navigate their ASCS and PSS SACE datasets, and to conduct focused and potentially useful analysis.

To support analysts through the entire analysis phase of the survey process, we have structured the analysis element of the MAX toolkit around two key activities:

1. Exploratory data analysis: establishing general trends in reported outcomes, measuring the impact of services on quality of life and identifying areas of potential interest or concern; and
2. Further analysis: conducting statistical and thematic analysis to answer local research questions and/or make sense of the general trends noted during exploratory analysis, to produce findings that can help you – and the decision-makers within your organisation – to better understand the views and experiences of adult service users and carers within your remit, identify priorities for local improvements and establish most appropriate strategies for amending and targeting services.

Survey specific guides have been produced for each activity and can be downloaded here. To enable users to quickly identify and access the most relevant and potentially useful resources, each guide includes links to the other tools included in the analysis element of the MAX toolkit. An overview of these tools is also provided on the restricted access statistical tests and tools page of the MAX toolkit (see screenshot below).

Screenshot from restricted access pages of the MAX toolkit

We have produced an overview presentation and a series of short blogs to help users make the most of the analysis element of the MAX toolkit. The next blog in this series discusses how ASCOF data from the surveys can be used to support local decision-making and service improvements, and can be accessed here.
____________________________________________________
Disclaimers
This blog is based on independent research commissioned and funded by the NIHR Policy Research Programme (Maximising the value of survey data in adult social care (MAX) project and MAX toolkit implementation and impact study). The views expressed in the publication are those of the author(s) and not necessarily those of the NHS, the NIHR, the Department of Health and Social Care or its arm’s length bodies or other government departments.
[1] Quote taken from MAX online survey (2013)