Tag: data

Straightforward data science intro

This looks to be an interesting response to the call to be more data literate. Via Flowing Data, a straightforward and potentially free way to get skilled up with R, without needing to install any software, it seems.

Chromebook Data Science – a free online data science program for anyone with a web browser – Simply Statistics
The reason they are called Chromebook Data Science is because philosophically our goal was that anyone with a Chromebook could do the courses. All you need is a web browser and an internet connection. The courses all take advantage of RStudio Cloud so that all course work can be completed entirely in a web browser. No need to install software or have the latest MacBook Computer.

Here’s some info on what the courses cover, including introductions to R and GitHub. Worth a look?

Excel’s getting interesting. No, really

News that Excel will soon be expanding its range of data types, enabling a much richer and more dynamic experience.

Excel Data Types
AI powered Excel Data Types will transform the way we work with Excel by enabling a cell to contain much more than text, numbers or formulas.

There are currently two Excel data types available to Office 365 users; Stocks and Geography. Let’s start with the Geography Data Type that can take a table of countries and return rich data that can be referenced in Excel formulas and expand into further columns.

excel-getting-interesting-2

Mynda takes us through many other examples of how these new data types can be used and referenced in our spreadsheets. And it seems like this is just the beginning.

The Excel team have big plans for Data Types with more coming, including the ability to create your own data types unique to your organisation. Imagine data types for Employees, Products, Stores, Regions… the list is endless.

“Reduced”?

Have I just been insulted by the head of Ofsted?

Spielman: Teachers ‘reduced to data managers’
Teachers have been reduced to “data managers” instead of “experts in their field”, Ofsted chief inspector Amanda Spielman will argue today. “I don’t know a single teacher who went into teaching to get the perfect Progress 8 score (a measure of pupil progress),” she will tell a schools conference in Newcastle this morning.

I see her point, though that’s unfortunate paraphrasing from TES. The Guardian’s version, after the speech actually took place, also has the line ‘reduced teachers to the status of “data managers”.’

Here’s the full, less patronising, quote, with no reduction to be seen.

Amanda Spielman speech to the SCHOOLS NorthEast summit
The bottom line is that we must make sure that we, as an inspectorate, complement rather than intensify performance data, because our curriculum research and a vast amount of sector feedback have told us that a focus on performance data is coming at the expense of what is taught in schools.

A new focus on substance should change that, bringing the inspection conversation back to the substance of young people’s learning and treating teachers like the experts in their field, not just data managers. I don’t know a single teacher who went into teaching to get the perfect progress 8 score. They go into it because they love what they teach and want children to love it too. That is where the inspection conversation should start and with the new framework, we have an opportunity to do just that.

And here’s another write-up, that doesn’t mention us data managers at all.

Spielman: Focus on substance over outcomes will help tackle workload
The chief inspector said she did not think there was an “appetite to revive the inspection model of 20 years ago”, but that the new framework, which will come into effect next September, will build on some of the “strengths” of the current system, “especially letting leaders tell their own story”.

“I also want to rebalance inspector time usage so that more time is spent on site, having those professional conversations with leaders and teachers, with less time away from schools and colleges in pre and post-inspection activity.”

Follow the data

I’m hearing more and more about data ethics. It wasn’t ‘a thing’ before, was it? But it certainly is now. Here’s a very interesting take on it: flow.

The ethics of data flow
In Privacy in Context, Helen Nissenbaum connects data’s mobility to privacy and ethics. For Nissenbaum, the important issue isn’t what data should be private or public, but how data and information flow: what happens to your data, and how it is used. Information flows are central to our expectations of privacy, and respecting those expectations is at the heart of data ethics.

It’s not what they’ve got, but what they do with it that matters.

The infamous Target case, in which Target outed a pregnant teenager by sending ad circulars to her home, is a great example. We all buy things, and when we buy things, we know that data is used—to send bills and to manage inventory, if nothing else. In this case, the surprise was that Target used this customer’s purchase history to identify her as pregnant, and send circulars advertising products for pregnant women and new mothers to her house. The problem isn’t the collection of data, or even its use; the problem is that the advertising comes from, and produces, a different and unexpected data flow. The data that’s flowing isn’t just the feed to the marketing contractor. That ad circular, pushed into a mailbox (and read by the girl’s father) is another data flow, and one that’s not expected.

[…]

Everyone who works with data knows that data becomes much more powerful when it is combined with data from other sources. Data that seems innocuous, like a grocery store purchase history, can be combined with geographic data, medical data, and other kinds of data to characterize users and their behavior with great precision. Knowing whether a person purchases cigarettes can be of great interest to an insurance company, as can knowing whether a cardiac patient is buying bacon.

The article is written by and for data developers, primarily, and poses more questions than it can answer, especially around the thorny concept of data deletion. It’s an interesting read, but it left me wondering if those GDPR data protection principles will ever be fully put into practice.

Happy statistics day

It’s GCSE results day and, despite the new grading system, the news people are bringing out updated versions of their usual it’s-getting-better-it’s-getting-worse stories.

GCSE results day 2018: New ‘tougher’ exams favour boys as gender gap narrowest in seven years
Girls remain in the lead, with 23.4 per cent achieving one of the highest grades, which is the same as last year, compared to 17.1 per cent of boys, up from 16.2 per cent last year. But the gap in top grades between boy and girls is now at its narrowest since 2010, with boys just 6.3 per cent behind girls, down from 7.2 per cent last year.

GCSE results rise despite tougher exams
A total of 20 of the most popular GCSE subjects in England have been graded for the first time in the numerical format – plus English and maths, which were introduced in the new format last year. These include history, geography, sciences and modern languages, all of which have been designed to be more difficult.

Of those achieving all grade 9s – and taking at least seven of the new GCSEs – almost two-thirds were girls.

GCSEs: boys close gap on girls after exams overhaul
Boys appear to have been the major beneficiary of the overhaul of GCSE examinations taken in England for the first time this summer, as results showed across-the-board improvements in boys gaining top marks while girls saw their share of top grades dip.

Across the UK the proportion of students gaining an A or 7 and above, the new top grade used in England, rose above 20%, with boys in England closing the gap on girls with an almost one percentage point rise to 17.1% with girls unchanged at 23.4%.

In the reformed GCSEs in England, 4.3% of the results were the new highest 9 grade, set at a higher mark than the previous A* grade. The figures on Thursday showed 732 students attained seven or more grade 9s.

Despite the improvements by boys in England they were still outperformed by girls at the highest level: 5% of entries by girls received 9s, compared with just 3.6% of boys.

GCSE pass rate goes UP – but fewer students get new top ‘9’ grade compared to old A* mark
The overall pass rate – the percentage of students getting a 4 or above or a C or above – was 66.9 per cent, compared with 66.4 per cent last year.

But just 4.3 per cent of exams were given the new 9 grade, which was brought in to reward the absolute highest achievers. Just 732 students in England got a clean sweep of seven or more grade 9s.

Previously around seven per cent of exams scored the top A* grade.

The more detail-oriented education sector websites are worth a read, if you really want to dig down into all this.

GCSE results 2018: How many grade 9s were awarded in the newly reformed subjects?
There has been a curious amount of interest in how many students might achieve straight 9s in all subjects. It seems to have started with a throwaway remark on twitter by the then-chief scientific adviser at the Department for Education that only two students would do so. Tom Benton from Cambridge Assessment then produced some excellent research showing that it would, in fact, be several hundred.

Today Ofqual has answered the question once and for all. A total of 732 students who took at least seven reformed GCSEs achieved grade 9 in all of them. Given that fewer grade 9s are awarded than grade A*, it should come as no surprise that fewer students will achieve straight grade 9s compared to straight grade A*s.

But it can get a little heavy-going at times.

GCSE results day 2018: The main trends in entries and grades
Across all subjects, 21.5% of entries were awarded a grade 7/A or above, compared to 21.1% last year. At grade 4/C or above, 69.3% of entries achieved the standard this year, compared to 68.9% last year. Both figures have been on something of a downward trend since 2015, so this year’s figures arrest this decline.

GCSE and A-Level results analysis
Explore trends in national entry and attainment data between 2014 and 2018 in:
All subjects
Additional mathematics
Additional science
Art and design subjects
Biology
Business and communication systems
Business studies
Chemistry
Citizenship studies
…

GCSE 2018 variability charts: Are your results normal?
Each year Ofqual produces boring-sounding variability charts. It sounds dull but they show how many centres, i.e. schools or colleges, dropped or increased their results compared with the previous year. This means that if you dropped, say, 25 per cent in one subject, you can see how many other schools also saw the same dip.

Let’s give the last word to the JCQ and Ofqual, and have done with it: I’m getting a headache.

JCQ Joint Council for Qualifications: Examination results
Each year, JCQCIC collates the collective results for its members from more than 26 million scripts and items of coursework. We only publish collated results from our members though and cannot supply regional, centre or candidate information.

Ofqual Analytics
Ofqual analytics presents a selection of data in an engaging and accessible way by using interactive visualisations. We hope this innovative approach to presenting data will make it easier to understand and explore the data we produce.

Map of GCSE (9 to 1) grade outcomes by county in England
The map shows reformed GCSE full course results (the percentage of students achieving specific grades) in England by subject and county for the summer 2018 examination series as well as the summer 2017 examination series. Data in the map represents the results that were issued on results day for both years (23 August 2018 and 24 August 2017) and do not reflect any changes following post-results services.

happy-statistics-day-2

(All I know is that my boy got his GCSE results today too, and they’re a credit to the amount of time and effort he’s put in over the years.)

AI to the rescue

In 2016 the RNIB announced a project between the NHS and DeepMind, Google’s artificial intelligence company.

Artificial intelligence to look for early signs of eye conditions humans might miss
With the number of people affected by sight loss in the UK predicted to double by 2050, Moorfields Eye Hospital NHS Foundation Trust and DeepMind Health have joined forces to explore how new technologies can help medical research into eye diseases.

This wasn’t the only collaboration with the NHS that Google was involved in. There was another project, to help staff monitor patients with kidney disease, that had people concerned about the amount of the medical information being handed over.

Revealed: Google AI has access to huge haul of NHS patient data
Google says that since there is no separate dataset for people with kidney conditions, it needs access to all of the data in order to run Streams effectively. In a statement, the Royal Free NHS Trust says that it “provides DeepMind with NHS patient data in accordance with strict information governance rules and for the purpose of direct clinical care only.”

Still, some are likely to be concerned by the amount of information being made available to Google. It includes logs of day-to-day hospital activity, such as records of the location and status of patients – as well as who visits them and when. The hospitals will also share the results of certain pathology and radiology tests.

The Google-owned company tried to reassure us that everything was being done appropriately, that all those medical records would be safe with them.

DeepMind hits back at criticism of its NHS data-sharing deal
DeepMind co-founder Mustafa Suleyman has said negative headlines surrounding his company’s data-sharing deal with the NHS are being “driven by a group with a particular view to peddle”. […]

All the data shared with DeepMind will be encrypted and parent company Google will not have access to it. Suleyman said the company was holding itself to “an unprecedented level of oversight”.

That didn’t seem to cut it though.

DeepMind’s data deal with the NHS broke privacy law
“The Royal Free did not have a valid basis for satisfying the common law duty of confidence and therefore the processing of that data breached that duty,” the ICO said in its letter to the Royal Free NHS Trust. “In this light, the processing was not lawful under the Act.” […]

“The Commission is not persuaded that it was necessary and proportionate to process 1.6 million partial patient records in order to test the clinical safety of the application. The processing of these records was, in the Commissioner’s view, excessive,” the ICO said.

And now here we are, some years later, and that eye project is a big hit.

Artificial intelligence equal to experts in detecting eye diseases
The breakthrough research, published online by Nature Medicine, describes how machine-learning technology has been successfully trained on thousands of historic de-personalised eye scans to identify features of eye disease and recommend how patients should be referred for care.

Researchers hope the technology could one day transform the way professionals carry out eye tests, allowing them to spot conditions earlier and prioritise patients with the most serious eye diseases before irreversible damage sets in.

That’s from UCL, one of the project’s partners. I like the use of the phrase ‘historic de-personalised eye scans’. And it doesn’t mention Google once.

Other reports also now seem to be pushing the ‘AI will rescue us’ angle, rather than the previous ‘Google will misuse our data’ line.

DeepMind AI matches health experts at spotting eye diseases
DeepMind’s ultimate aim is to develop and implement a system that can assist the UK’s National Health Service with its ever-growing workload. Accurate AI judgements would lead to faster diagnoses and, in theory, treatment that could save patients’ vision.

Artificial intelligence ‘did not miss a single urgent case’
He told the BBC: “I think this will make most eye specialists gasp because we have shown this algorithm is as good as the world’s leading experts in interpreting these scans.” […]

He said: “Every eye doctor has seen patients go blind due to delays in referral; AI should help us to flag those urgent cases and get them treated early.”

And it seems AI can help with the really tricky problems too.

This robot uses AI to find Waldo, thereby ruining Where’s Waldo
To me, this is like the equivalent of cheating on your math homework by looking for the answers at the back of your textbook. Or worse, like getting a hand-me-down copy of Where’s Waldo and when you open the book, you find that your older cousin has already circled the Waldos in red marker. It’s about the journey, not the destination — the process of methodically scanning pages with your eyes is entirely lost! But of course, no one is actually going to use this robot to take the fun out of Where’s Waldo, it’s just a demonstration of what AutoML can do.

There’s Waldo is a robot that finds Waldo

We all need to be data literate

This article from Harvard Business Review doesn’t mention schools once, but I think it fits perfectly well in that setting.

The democratization of data science
Intelligent people find new uses for data science every day. Still, despite the explosion of interest in the data collected by just about every sector of American business — from financial companies and health care firms to management consultancies and the government — many organizations continue to relegate data-science knowledge to a small number of employees.

That’s a mistake — and in the long run, it’s unsustainable.

It goes on to outline the three steps necessary to create a more data literate organisation; share data tools, spread data skills, and spread data responsibility. Couldn’t agree more. It’s well worth a read.

Excel tips and tricks

Often with these kinds of training videos it’s the ‘ooh I’d forgotten you could do that‘ moments that are the most useful. The overall topic of the tutorial itself might not be what you’re after (do you really need to know how to make a Christmas Tree in Excel?), but just simple reminders about things like F9 or index match are just the ticket.

Here are some of my recent favourites:

From Exceljet‘s channel, How to fill in missing data with a simple formula

From Leila Gharani‘s channel, Excel index match advanced: lookup multiple criteria in rows or columns

From the My Online Training Hub channel, Labeling events in Excel charts

From the Excel Is Fun channel, Excel magic trick 1474: Excel twinkling Christmas Tree with star & formulas as presents!

Nicholas Felton's new app, Reporter

reporter

Nicholas Felton, the man behind the Feltron Reports and Daytum, has a new app out, Reporter. He says on his blog

“Reporter’s random prompts to answer a survey had made tracking the year a breeze and helped me to investigate questions that would have been impossible to answer using other methods. I was interested in who I spent time with, but to track this in an ongoing basis is a full-time job. I added questions for what I was wearing, eating or drinking and if I was working or not… and we streamlined the process to ensure that a report only took seconds to answer. We also added in background sampling to get information from the phone on the weather, my location and the ambient noise level.”

I loved Daytum and at one point was using it quite extensively. I wanted to use it to track which of my ties are my favourites, but couldn’t find a way of neatly naming them (the blue-ish purple-ish TM Lewin one, the more regimented gold-ish Van Buck one). I can’t see this new app helping with that question especially, but it’ll be fun to do the quantified self thing again for a while.

Still far from clear(ing)

Andrew Fisher goes over the latest UCAS data; things are still far from clear

Consider the case of the applicant who just misses the AAB threshold. Prestigious institutions which have had many of their AABs hijacked by even more prestigious ones (maybe SOAS) and would therefore like to have this person, may not be able to accept such students at Confirmation or in Clearing, because they will be constrained by the  SNC; so there may well be a population of applicants too proud to accept the institutions able to accept them.  Even students well short of AAB may not be willing to go to certain institutions in Clearing. Once London Metropolitan is full at £6k or less, will applicants be willing to pay £9k to go to UEL or will they decide that at that price it is better to miss out on HE? We don’t know.

Read the rest and check out the charts. As he says, “the proof of the pudding will be in the Clearing.”

HE standards, email

Watchdog finds fault with Leeds Met validation
The Quality Assurance Agency has raised concerns about Leeds Metropolitan University’s validation of degrees. The watchdog said this week that it had “limited confidence” in the institution’s management of academic standards for courses delivered by partner colleges.

The price of a University drop-out 4: Time for some numbers
The data takes into account HEFCE decided partial completion premiums and the reduced funding delivered for each masters and post-graduate student (because they pay entirely for their course), and is, basically, quite complicated. The data I compiled (in slightly raw, Google doc form) was drawn from HEFCE-released data from 10/11, and is free for anyone to play with, so if you want to see how your university fares for non-completion rates, take a look. So what can we take from this? I think the most interesting discussions will be for the future, and how the whole system will dramatically change why tuition fees sky-rocket.

A “zero email” policy that makes zero sense
According to this article (also covered by the WSJ), the French IT company Atos has discovered that its employees are becoming less productive because of the increasing onslaught of email. … the CEO announced that the company will BAN EMAIL. This is a technology company with 74,000 employees. No more emails – internally, at least, as a few people outside the company still use the tool. If you work in X business, shouldn’t you make sure your employees are good at X?

Twitter strategies, visualising data, managing projects

Template Twitter strategy for Government Departments
You might think a 20-page strategy a bit over the top for a tool like Twitter. After all, microblogging is a low-barrier to entry, low-risk and low-resource channel relative to other corporate communications overheads like a blog or printed newsletter. And the pioneers in corporate use of Twitter by central government (see No 10, CLG and FCO) all started as low-profile experiments and grew organically into what they are today. But, having held back my JFDI inclinations long enough to sit down and write a proper plan for BIS’s corporate Twitter account, I was surprised by just how much there is to say – and quite how worth saying it is, especially now the platform is more mature and less forgiving of mistakes.

50 great examples of data visualization
50 of the best data visualizations and tools for creating your own visualizations out there, covering everything from Digg activity to network connectivity to what’s currently happening on Twitter.

University of Edinburgh Records Management Section – advice on freedom of information, data protection and records management
The Records Management Section provides help and advice to all units of the University on information management issues including records management practices and procedures, data protection and freedom of information.  We are also responsible for the Central Records Registry and the day-to day management of the records of the central administrative areas formerly known as Policy and Planning.

University of Edinburgh Projects Web Site
A one stop source of information about University IT projects – Templates and methodologies to assist in the successful management of projects – A filing system and repository for project related documentation – A communication vehicle for keeping stakeholders informed about project progress.

HE records, vagueness

Records Retention in HE Introduction
JISC infoNet is pleased to announce the launch of the new Business Classification Scheme (BCS) and Records Retention Schedule (RRS) for Higher Education Institutions. The 3rd iteration of this highly regarded resource is the result of extensive consultation within the sector plus significant additional research both undertaken for JISC infoNet by Emmerson Consulting.

Book of the week: Not Exactly: In Praise of Vagueness
In an increasingly complex digital world, it is tempting to get sucked into the precision – often a wholly spurious precision – that seems endemic to this culture. I recently caught myself doing it, despite my own distinctly analogue leanings. When asked for the time at a bus stop last week, I found myself replying “Seven forty-eight” – instead of the answer I would usually give: “Just after quarter to eight.” I put this momentary lapse of good taste down to early morning low blood sugar and the disabling effect of public transport, but it is a symptom of something much bigger – a subject that has captured the (not insubstantial) imagination of computer scientist Kees van Deemter.

Data gov

Data.gov.uk versus Data.gov – Which wins?
Back in May last year, the US government launched Data.gov as a statement of transparency, and the Internet rejoiced. After the launch, excitement kind of fizzled with the actual Data.gov site. Then just a couple of weeks ago, Data.gov.uk launched, which brought me back to the US counterpart. How do the two compare? Here’s my take.

Disappointment, digital design

Why are you so terribly disappointing? – SF Gate
What happened to my bonus? What happened to my job? What happened to my country? Why can’t it all go the way it’s supposed to go? You mean having a kid won’t solve my marriage problems? Why don’t these drugs make me feel better? Where’s that goddamn waiter with my salad? Have you seen the stupid weather today? Is this really all there is?

Making digital content on the mobile phone physically graspable – Infosthetics
The weight-shifting method allows a phone to communicate to users where to walk by dynamically changing its gravitational center along two axes. The shape-changing method is able to convey where more information is located outside of the screen by changing the thickness of a phone at its corners. And lastly, the ‘living’ method allows a mobile phone to display emotional states due to a continuous heartbeat and breathing-like motion that can be felt ambiently in your trouser pocket.

Privacy, iPads, art student satisfaction

What day is it? It’s Data Privacy Day!
Take a moment and think about what Google knows about you. Correspondence and contacts via email, schedule via calendar, interest via feed reader, purchases via Checkout, and most importantly your day-to-day via search. How do you feel about a single company knowing that much about you? Don’t you want to know how they use all that data and more importantly, how they protect it?

Diagnosing the tablet fever in higher education
Tablet-style computers could be game-changers for colleges, bringing in a new era of classroom collaboration and pushing the adoption of electronic textbooks over a tipping point. Today’s announcement by Apple Inc. of the iPad tablet has education watchers predicting a wave of student purchases, major textbook publishers rejoicing, and at least one college saying it will consider giving them to all incoming students. But wait — it might be time to take a deep breath to let the excitement of the sales pitch fade.

Academics in art and design have drawn up a plan to tackle stubbornly low scores for student satisfaction in the National Student Survey
[D]espite efforts to improve the ratings, art and design still does not perform well in comparison with other subject areas, according to a forthcoming report, I Can’t Believe It’s Not Better: The Paradox of NSS Scores for Art and Design. […] The widely held view was that the pedagogy of art and design subjects, where students are encouraged to explore and navigate their own way through projects with support, was poorly served by NSS questions, which were felt to relate more to subjects with a highly timetabled, often lecture-based, structure.

Managing time, visualising proportions

Testing time-management strategies
I asked a half-dozen executive coaches to help me pick the most widely used time-management systems—not just software tools or high-tech to-do lists, but behavioral-change techniques that help people get organized, clarify thinking and increase output. Then, I tried out for a week each of the three methods they mentioned most often—including one that involved a ticking plastic tomato.

The Pomodoro Technique™
The Pomodoro Technique™ is a way to get the most out of time management. Turn time into a valuable ally to accomplish what we want to do and chart continuous improvement in the way we do it.

Anything that uses lists, little boxes, ‘x’s and apostrophe markers can only be a good thing, right?

9 ways to visualize proportions – a guide
With all the visualization options out there, it can be hard to figure out what graph or chart suits your data best. This is a guide to make your decision easier for one particular type of data: proportions.

The pie, the donut, the stacked area, the stacked bar, the treemap, the voronoi, the nightingale (my new favourite) and the everything, all with links to real-world examples.