目錄

Introduction 1.1

Introduction 1.2

License 1.2.1

Preface 1.2.2

Foreword 1.2.3

How to Use This Book 1.2.4

Data Fundamentals 1.3

Basic Data Types 1.3.1

About Data Aggregation 1.3.2

Collecting Data 1.4

Intro to Survey Design 1.4.1

Types of Survey Questions 1.4.2

Additional Data Collection Methods 1.4.3

Finding External Data 1.4.4

Preparing Data 1.5

Getting Data Ready for Cleaning 1.5.1

Data Cleaning 1.5.2

Types of Data Checks 1.5.3

What Data Cleaning Can and Can’t Catch 1.5.4

Data Transformations 1.5.5

Visualizing Data 1.6

Deciding Which and How Much Data to Illustrate 1.6.1

Graphing the Results of Checkbox Responses 1.6.2

Anatomy of a Graphic 1.6.3

Importance of Color, Font, and Icons 1.6.4

Print Vs. Web, Static Vs. Interactive 1.6.5

What Not To Do 1.7

Perception Deception 1.7.1

Common Visualization Mistakes 1.7.2

Conclusion 1.8

Resources 1.8.1

Glossary 1.8.2

Contributors 1.8.3

Introduction

資料＋設計

資訊準備與視覺化的簡單入門

這是正在進行中的翻譯計畫，歡迎加入一起填坑！ bit.ly/data-design-zh

以下取自原作。

關於本書

資訊設計是為了弄懂資料.

無論你是在為報紙寫文章、展示系列運動成果、介紹你的學術研究、說明你們團隊的績效指標，還是闡述公共議題，你都需要知道如何呈現你的資料，以便讓別人可以弄懂它。

不管你用什麼工具來蒐集資料進而視覺化，從作者角度而言，為了說個好故事，你需要對主題和資料集做出決定。因此，你需要了解蒐集、清理，以及視覺化資料的主要課題。

這本自由的創用 CC 授權電子書用簡單的話說明重要的資料概念。可以把它看做是一個深入的資料常見問答集，讓想得到額外幫助的平面設計師、內容製作者，與不太精通技術的鄉民們知道從哪裡開始，以及視覺化資訊時要注意什麼。

這一切的背後有哪些人？

這本書是由全球十四個國家超過50個志願者所精心撰寫。

《資料＋設計》也是 Infoactive 和the 唐納德·W·雷諾茲新聞學院（Donald W. Reynolds Journalism Institute）之間的協作。

在 Infoactive，我們的目標是讓人們更容易地運用資料來說互動故事。我們相信，你不需要成為一個設計師、開發者，以及資料分析師才能夠做出很棒的資訊圖表。透過致力於簡化資料說故事的過程，我們了解進行資料相關工作時，說簡單的話有很高的需求——尤其是對於那些被認為數學不是很好的人。

藉由通俗語言書寫及相關實例的輔助，我們準備將 Infoactive 簡單、無痛的精神融入這個專案。我們也希望最終的結果成為每個人的資源，無論他們使用什麼軟體。

雷諾茲新聞學院（The Reynolds Journalism Institute, RJI） 目標在幫助新聞工作者。RJI 贊助此電子書專案，因為資料視覺化是新聞工作者講故事極好的工具。

現在就參與

這是一個開放原始碼、協作的專案 ，我們歡迎你的參與！原始碼可以在 GitHub 取得， 連同授權與貢獻此專案的基本指示。

我們有些貢獻者在討論把這本書翻譯成其他語言、增加更多的章節，甚至撰寫第二版的《資料＋設計》 。這本書的貢獻者聰明、友善，各方面合作起來都很棒，所以我們確信，這本書的下一個階段將為所有參與者帶來很大的樂趣。

要寫新的章節？成為志願編輯？推出第二版？回報錯字或錯誤？ 請用這份表格讓我們知道。

這個專案之所以成立，是由於超過50位來自世界各地14個不同國家的志願者聚集在一起。我們認為這是相當驚人的，而我們要感謝所有過去和未來促成這件事的貢獻者！

新聞聯繫

想寫篇關於本書的報導，請查閱我們的新聞資料袋並聯繫電子郵件尋求更多資訊，或是安排採訪。

Chicago Write-A-Thon Helps Storytellers Make Sense of Data

連結

《資料＋設計》不只是一本書。這是一個使資料易於理解及樂於探索的全球性運動。我們很高興你在這裡，我們非常歡迎你與我們聯繫打聲招呼。

@DataDesignBook

電子郵件

Introduction

資料＋設計

資料準備與視覺化的簡單入門

由 Trina Chiasson、Dyanna Gregory，及所有這些人所著。

經由 Infoactive 以及唐納德·W·雷諾茲新聞學院（Donald W. Reynolds Journalism Institute）所支持。

License

資料＋設計

資料準備與視覺化的簡單入門

資料＋設計正體中文版由 g0v 貢獻者翻譯，以創用CC 姓名標示-非商業性-相同方式分享 4.0 國際 授權條款釋出。
此作品衍生自 Data + Design，由 Trina Chiasson、Dyanna Gregory，及所有這些人所著，經由 Infoactive 以及唐納德·W·雷諾茲新聞學院（Donald W. Reynolds Journalism Institute）所支持。

我們樂意見到你改作、重混，並改善本著作的非商業專案。</p>

超過50人在許多個月的努力下創作了本書，我們很高興將它釋出為自由資源。如果你使用或改作本著作，請確保對實現它的貢獻者進行標示，包括給予 g0v.gitbooks.io/data-design 及 infoactive.co/data-design 的連結。

原始碼可以在 Github 取得並岔出。

Preface

Preface 作者序

I hadn’t written a book before, let alone an international, open-source book with more than 50 contributors from 14 different countries.

我之前沒有寫過書，更別說一本橫跨 14 國、多達 50 名貢獻者的國際開源書籍。

It started with a message on Kickstarter:

這一切開始於一則在 Kickstarter 上的留言：

Hi Trina! Stats dork from Chicago here….Do you have any plans to include tutorials for basic data cleaning and data selection techniques for users who may not have any statistics background?

Trina 你好！我是一個住芝加哥的統計宅 …。你們未來有沒有計劃替沒有任何統計背景的的人，寫一些基礎資料清理與資料遴選技巧的教學文呢？

At the time, I didn’t know that this one message would turn into a book, a community, and a global endeavor to make information design more accessible.

那時的我，並不知道這則留言會變成一本書、催生一個社群，甚至點燃不同國家的人的熱情，有志一同地讓資訊設計更加親民。

The message author, Dyanna Gregory, was a statistical programmer who knew the challenges of teaching university-level stats to those who don’t identify as math nerds. I was an entrepreneur building Infoactive, a web application to help people create interactive infographics and data visualizations. I was also a Reynolds Fellow at the Donald W. Reynolds Journalism Institute where my goal was to find ways to simplify the process of making data visualizations in newsrooms. I had launched a Kickstarter campaign to support Infoactive, and we had nearly 1,500 backers who were excited about breaking down barriers in data visualization.

那則留言的作者 Dyanna Gregory 是一名統計學程式設計師，深知將大學統計學教給數學宅以外的人，是多大的一個坑。我則是一名連續創業家，打造 Infoactive 網路應用程式，幫助人們創造互動型資訊圖表與資資料視覺化。我也曾是美國密蘇里大學 Donald W. Reylonds 新聞學院 (RJI) 的 Raylonds Fellow，研究如何簡化新聞編輯室裡做資料視覺化的流程。我在 Kickstarter 上替 Infoactive 募資，有近 1500 人贊助；贊助者都十分樂見資料視覺畫的門檻能變得更低。

We all believed in the vision of making data simple.

我們都懷抱著讓資料更簡單的願景。

But working with data can be far from simple. Data come in all different shapes, sizes, and flavors. There’s no one-size-fits-all solution to collecting, understanding, and visualizing information. Some people spend years studying the topic through statistics, mathematics, design, and computer science. And many people want a bit of extra help getting started.

但處理資料可沒有那麼容易。資料有著各種奇形怪狀，既能搜集、理解又可以將資料視覺化的萬用必殺技並不存在。有些人研讀統計、數學、設計與電腦科學好幾年，就是為了鑽研這個主題；也有很多新手想踏入這個殿堂，需要一些小小的幫助。

Dyanna and I began talking about what a plain-language data resource would look like. Usability was a big priority for us. It’s hard to write a technical book that’s easy for less-technical readers to digest, but we believed that it was an important challenge to tackle. We wanted to create a free resource that was well-designed, joyful to read, and easy to understand.

我跟 Dyanna 開始討論如何用淺白的語言來談資料。易讀性至關重要；一本給非技術讀者消化的技術書籍確實不好寫，我們相信這是必須要去面對的重要挑戰。我們想要創建一份免費／自由的參考資源，它設計精良、能輕鬆閱讀，也易於理解。

Of course, the information would need to be accurate and we wanted to cover a range of different data concepts. We needed technical writers with an in-depth understanding of data, math, and statistics. We also needed editors who could comb over the content to make adjustments for simplicity and understandability and ensure that the chapters had a friendly, conversational tone. Our mission was to translate geek into non-geek — to make technical concepts more accessible to less-technical people.

當然，這份教學資訊仍需精確到位，我們也希望能囊括不同的資料處理概念。我們需要技術背景的寫手深入介紹資料、數學與統計。我們也需要爬梳並調整內容、讓內容更好理解的專業編輯，確保各章節能保持友善的對話語調。我們的使命是化宅為非宅 — 讓艱深的技術概念更容易被非技術人所理解。

Dyanna and I made a call for contributors. Neither of us expected to see such a strong, positive response. Frankly, it blew our minds. Messages began pouring in from people from all over the globe who told us about their experiences working with data and design, or their lack thereof. What struck me the most was the number of self-identified “non-math people” who were hungry for a resource that could introduce them to data concepts in a manner that was simple, approachable, and even fun. They were motivated to pitch in and volunteer their time to make it happen.

我跟 Dyanna 開始徵求貢獻者加入。大出所料的是，我們收到了大量的正面迴響；說實話，這讓我們沖昏了頭。來全世界的留言湧入，告訴我們他們如何處理資料與設計，或他們如何地缺乏這些能力。最讓我震驚的是，有非常多自稱為「非數學人」的人們，非常渴望能有一份教學資源，用簡單易懂甚至有趣的方式來介紹資料處理的概念。這些人們自願投入時間，協力讓這美好的事情發生。

Dyanna and I kicked off the project in February with a Write-A-Thon in Chicago. We invited writers, data analysts, programmers, designers, and others to come together in person and talk about the project as a whole. We thought through the process, talked about data, opened up our laptops, and started writing.

二月，在一場芝加哥的作家松（Write-A-Thon）中，我們開始了這個計劃。我們邀到了一些作家、資料分析師、設計師、程式設計師等人，面對面地討論整個計劃。我們在腦海裡順過整個流程、討論一下資料，就打開筆電動手寫書。

After that, we contacted everyone who applied to contribute, and began figuring out who was going to do what. For an all-volunteer project with more than 50 contributors spread across many time zones, it was important to stay organized. We found project managers for different sections of the book, chose writers and editors for each chapter, and put together research and distribution teams. Chapter by chapter, the vision began to turn into a reality.

在那之後，我們向報名貢獻的人聯絡，並且開始分工。這個全自願性專案有超過 50 個貢獻者、橫跨不同時區，因此管理變得很重要。我們將整本書的不同區段指派給各專案經理、選擇各章的責任作家與編輯，並且設置了研究與散佈小組。一章接著一章，願景逐漸地化為真實。

For me, the best part of this project was having the honor of working with some of the smartest, funniest, most creative people I’ve ever met. Again and again, I’ve been blown away by their creativity, passion, and support. I’m beyond grateful that I had the experience of creating this book with such an incredible group of people. Data + Design is truly a community effort.

對我來說，參與這個專案最棒的地方，在於能有榮幸與我所遇過的一些最聰明、最有趣、最有創意的人們一起共事。他們的創意、熱忱與支持一次次地驚呆了我。我非常感激能有這個機會與這些不可思議的人們一起寫這本書。Data + Design 完全是社群的共同努力。

This book isn’t a final product. It’s the beginning of a community process to improve our collective understanding of data and design. We’re releasing the first edition now, but we’re already working on more chapters for future releases and thinking about ways that we can improve. Together we can build upon it, translate it, transform it, and make it better with every iteration.

這本書並非成品，而是社群對話的開始，精進我們對資料與設計的集成智慧。雖然現在才剛發行第一版，但我們已經在準備未來會釋出的新章節，也在思考如何更進步。以此為基礎，我們能一同創建、翻譯、轉化，透過一次次地改版，讓它更好。

Data + Design is open source and available on Github. It’s free for anyone to read, download, remix, and re-distribute for noncommercial purposes. We invite you to join us. Email ebook@infoactive.co to get involved.

Data + Design 完全開源，你可以在 Github 上找到它。任何人都可以自由／免費地閱讀、下載、改作，與不帶商業目的地重新散佈。我們邀請你加入我們，請來信到 ebook@infoactive.co。

I also want to thank the Donald W. Reynolds Journalism Institute (RJI) for supporting Data + Design. RJI’s support for journalism and data storytelling played an instrumental role in bringing this project to life.

我也想要感謝 Donald W. Reynolds 新聞學院 (RJI) 支持 Data + Design. RJI 對新聞與說故事技巧的支持對這個計劃的誕生至關重要。

Trina Chiasson
Co-founder & CEO, Infoactive
2013-2014 Reynolds Fellow

Foreword

Foreword

資料遍佈生活周遭，從古迄今皆是如此。歷史載明的一切都有量化的可能：理論上，可以算數曾活在世上的人們，心跳數，曾經踩過的步伐，曾經閃耀過的星星，曾經口語表達或書寫紀錄的字詞。每一個被歷史撿取記載的事物都能以數字呈現。直到不久前，吾輩才擁有技術將這些隱藏的數字予以有效果地呈現，導引我們至人類處境的深刻洞見。

這到底意謂著什麼？容易接受資料的文化影響為何？這表達了一件事，我們需要更進一步資料識讀的能力。這也意謂我們必須更多設計識讀的能力。俗諺曾謂，統計會說謊。嗯，資料視覺化也同樣會說謊。首先，如何學會有效判讀視覺化的資料？其次，如何以合乎倫理且清楚的方式傳達資料所包涵承繼的內容？

At the intersection of art and algorithm, data visualization schematically abstracts information to bring about a deeper understanding of the data, wrapping it in an element of awe. 「計算與藝術的交融互涉，將資料視覺化是將資訊約略抽象帶著敬畏的態度將其包裝整飾，讓人能夠更深入的理解這些資料。」 Maria Popova， Stories for the Information Age, Businessweek。

Maria Popova, Stories for the Information Age, Businessweek

My favorite description of data visualization comes from the prolific blogger, Maria Popova, who said that data visualization is “at the intersection of art and algorithm.” To learn about the history of data visualization is to become an armchair cartographer, explorer, and statistician. 對於資料視覺化的描述，我偏好善寫的部落客Maria Popova所說的「計算與藝術的交融互涉」。要瞭解資料視覺化的歷史，需要成為扶手椅上的製圖師、發掘者和統計學家。

Early visual explorations of data focused mostly on small snippets of data gleaned to expand humanity’s understanding of the geographical world, mainly through maps. Starting with the first recognized world maps of the 13th century, scientists, mathematicians, philosophers, and sailors used math to visualize the invisible. Stars and suns were plotted, coastlines and shipping routes charted. Data visualization, in its native essence, drew the lines, points, and coordinates that gave form to the physical world and our place in it. It answered questions like “Where am I?”, “How do I get there?”, and “How far is it?” 早期資料的視覺探索大部分聚焦於資料的細碎片段，這些資料當初收集的目的是擴展對世界的理解，主要的手段是地圖。從十三世紀的世界地圖開始，科學家、數學家、哲學家以及水手運用數學將不可見變為可見。星辰日月可以，海岸線與航線可以圖示。最初的特質而言，資料視覺化繪製線條、標點及標示座標，賦予了物理世界以及身處的地方視覺化資料的形式。這回答了了諸如「我在哪？」「我要如何到那？」以及「那有多遠？」

Early data visualizations were also used to answer questions pertaining to issues of public health. Epidemiologist John Snow’s 1854 London cholera map was created to record instances of cholera in a London neighborhood, pinpointing the cause of the outbreak to a single well. This knowledge gained from patterns in lists of names, numbers, and locations was then used to persuade London’s populace to install sewer systems to alleviate the proliferation and spread of disease. The human brain is particularly adept at recognizing patterns, and a good data visualization, like Snow’s, optimizes displays of these patterns through effective use of Gestalt theory, design principles, and color. (Or lack of it, as this case may be.)

Snow’s visualization, with its absence of color, optimizes Gestalt’s theories of visual perception, most notably “Proximity” and “Figure and Ground.” The small black dots, each one representing a single case of cholera are small black figures standing out in contrast against the ground: in this graphic, the lines and white space representing streets. The proximity of these dots around the affected well are what enabled Snow to determine the exact source of the outbreak. Today, even with our advanced computing systems and sophisticated tools for creating data visualizations, there is little you could do to improve the effectiveness of this chart. It is simple, beautiful, and true: a data visualization that saved lives.

Florence Nightingale, famous more for her nursing skills than her analytic prowess, was nonetheless also a master data scientist and storyteller. Through data presented via her signature Coxcomb diagram (also known as polar or rose charts), she convinced the British army to invest in sanitation measures after illustrating that the majority of deaths in the Crimean War were the result of preventable diseases caused by the horrible sanitary conditions in hospitals. “Why are we sick?” she asked, then answering the question herself by giving visual form to data.

Looking at this graph, it is readily apparent that preventable diseases outnumbered all other causes of death. The area in blue represents deaths by preventable diseases, measured from the center, with red representing deaths caused by injuries and black indicating all other causes. Design principles at play here include the addition of color theory to take advantage of more Gestalt principles: “Similarity” and “Continuity”. Color makes it easy for us to tell which segments belong to which category. It also helps to draw the eye in a continuous path around the graphic, making it easier to read.

There is debate over the quality of this chart. Some claim it one of the best, most memorable visualizations ever created, not solely because of its visual communication strength, but in spite of it. It is remembered because of the change it inspired. Others deride it, claiming it’s just a glorified pie chart, suffering from the same misrepresentation of the information by distorting the data: in terms of visual perception, humans have a hard time accurately judging measures represented by differences in area. Despite their ubiquity, pie charts, for this very reason, are an incredibly poor way to visualize data. A simple stacked bar chart with reference lines, while not as beautiful or visually intriguing, would have communicated more effectively and on a quicker read.

We still ask the same kinds of questions that Snow and Nightingale posed, and as curious humans, probably always will. But the major differences today are that we have the means with which to gather much larger, richer datasets. And we also have the tools with which to automate visualizing our data.

This isn’t to say that small datasets, like the ones Nightingale and Snow observed, are any less valuable or interesting. Thanks to data gathering technology and advances in wearable computing and the Internet of Things, to the contrary. My friend Jen Lowe, a data scientist and artist working in New York, recently published her heartbeat on the internet. As a flat, static visualization, it would be beautiful but not especially informative. But by adding interactivity and live data feeds of her pulse via wearable sensors and API calls, her heartbeat is a living, beating, digital thing, viewable by anyone, anywhere, at any time. What you have is insight into another human being like you’ve never seen before.

Unique insight is the essence of data, both big and small, and the result of the tools that allow us to access, probe, poke, prod, dissect, visualize, and hopefully, make sense of it. Tools which, through the democratization of data visualization, allow us to change our lens on the world, creating pictures of humanity from different perspectives, bringing into focus stories about humanity and the world that were previously invisible, allowing us insight into ourselves like we’ve never seen before.

Callie Neylan
Senior Designer, Microsoft

How to Use This Book

How to Use This Book

Structure of the Book

The book’s chapters follow pretty closely the actual set of steps that need to be accomplished before data can be visualized, from the design of the survey to the collection of the data to ultimately its visualization. Really pretty simple. So if you’re fairly new to the game, it is probably going to make sense for you to start at the beginning and read the whole way through the book. Likewise, if you’re familiar with some of the content, but not everything, there’s no harm in just skipping to the chapters that are relevant to your case: you aren’t going to be scrambling to catch up if you skip around!

If you don’t have much (or any) experience with survey design and data collection and visualization, it would probably be best to read this book all the way through before you start collecting your data. Some of the visualizations you want to make when everything is said and done might require some well-planned decisions up front. If you’re reading this book part-by-part as you go, there’s a better chance that you might not consider something until it’s too late!

Common Terms and Definitions

Throughout the book, there may be times when we introduce specialized terms and phrases that are rather specific to the topic being discussed. Whenever this is the case, we will provide a link to the term in our glossary. You’re welcome to reference this at any time, but we do our best to point you in that direction when a new term is used.

Advanced Chapters

This book does contain some chapters that go a bit above and beyond just the basics that we think you should know. We do denote these chapters with a warning at the beginning of the chapter so that you know to expect a more in-depth look at the topic. Reading these chapters (although we recommend it!) is not necessary for an adequate understanding of the broader topic being discussed; however, there are times when we feel that it’s useful to provide you with easy access to the information and that you could benefit from a more thorough understanding of the topic.

Examples: The Good, the Bad, and the Ugly

One of our favorite parts about this book (although maybe we’re a little biased) is the examples that we use throughout: many of these concepts can get kind of abstract and having a concrete example to refer to really helps to keep things straight. That said, there are a few different styles that we use to indicate good and bad examples, dos and don’ts, etc.

Question | Answer
What are we hoping to find? | The answer lives here.

Good Examples

Good examples are always accompanied by a green or blue bar, a checkmark, or a thumbs-up symbol, depending on what’s being presented (a graphic, a data table, a procedure, something else).

Bad Examples

Conversely, whenever we illustrate what not to do in a given situation, we will include a gray bar or cross out the example with a large X or include a thumbs-down symbol to accompany it. These different styles will look like:

Pros | Cons
The benefits of an action go here. | And the negatives go here.

Do | Don't
This column lists good tips and advice. | This column shows what you should avoid.

Sometimes we also mark good and bad examples using images

An image preceded by a gray box with an "x" means that it's a graphical example of a practice that you should avoid.

This is an example of what not to do

Warnings

Other times, we will just want to make sure that you’re careful and don’t forget to consider an important point. When this is the case we will include a warning box, identified by a red exclamation mark. (Actually, we already used one of these up above!)

Making Contributions and Corrections

If you’re reading through the book and notice a typo, or think that something is incorrectly or ambiguously explained, or get a great idea for a new section or chapter, let us know! There are two ways to do this: you can either fill out this form or, if you’re a little bit more technical, follow the directions that we have on GitHub.

Likewise, if you just aren’t sure about something after reading about it, ask us for a better explanation! It’s likely that we didn’t realize that we could have explained it better (but that doesn’t mean we don’t want to!). Tweet us at @DataDesignBook, email ebook@infoactive.co, or check out HelpMeViz for even more resources. We’ll do our best to get your question squared away and will make sure that future editions do a better job addressing the issue.

Data Fundamentals

資料的基礎

當你在烹調佳餚的時候，你會按照特定的順序去完成每個步驟。找出需要哪些材料，並準備這些材料，之後料理一番並展現成品。

其實資料視覺化跟料理非常地相似。你將決定會使用到哪些數據資料，將其搜集、整理備用，最後再將其視覺化並展示成果。

而當你在料理的時候，熟知每樣材料能夠讓你做出一道美味的飯菜。譬如説鹽和糖看起來相似，但用來調味卻會有截然不同的效果呢！

在這一章節裡，我們會談到資料的基本成分，以及數據資料能夠和不能達成什麼，讓你在稍後能更有概念地去處理這些資料。

Basic Data Types

第一章

基礎資料型態

作者: Michael Castello

瞭解你能如何處理許多不同型態的資料相當重要，這樣你才能為你的需求找到蒐集資料的最佳方式。人們以許多不同的方式來詮釋資料，在這裡我們主要使用四種測量尺度(level of measurement)，包含名目型(nominal)、順序型(ordinal)、區間型(interval)以及比率型(ratio)。

測量尺度

想像你正在雜貨店中購物，一邊穿梭在貨架間，一邊將需要的商品放入購物籃中，裡面已經裝了一些鮮食、乳製品、冷凍食品以及罐頭，這時如果你想列一個清單整理這些商品來自於哪個貨架，這些資料將屬於名目型。"nominal"這個單字出自於拉丁文中的"nomen"，意思是"關於名稱的"，因此由個別資料以及其所屬的類別名稱所組成的資料，我們稱之為"名目型資料"。名目型資料是無序的(unordered)，舉例來說，我們不能將一個產品的分類與奶製品在數學上做大小的比較。

名目型

名目型資料可數且常用來計算百分比，但不能計算其平均數，就像我們能指出購物籃內的多少商品或多少比例的商品是來自乳製品區，但卻不能計算籃中商品中來自哪個區域的平均值。

當有些資料只能被分為兩類，我們稱它為二元(dichotomous)。一個只能回答"是"或"否"問題的答案是二元資料，比如我們在購物時，蒐集產品有無特價的資料將屬於一種二元資料。

順序型

最後，你準備結帳並想找一排能夠最快速結完帳的隊伍，沒有仔細地計算每一排隊伍有多少等候的顧客，你大略地將隊伍分成短、中、長三種，這種資料除了類別以外還有天生的順序性，我們稱之為順序型資料(ordinal data)。在問券中，會以選項是"非常同意"、"同意"、"中立"、"不同意"、"非常不同意"的問題來蒐集順序型資料，雖然我們常給予這些選項一個對應的數值來方便分析(例如以1代表非常不同意，5表示非常同意)，但這些數值並沒有數學上的意義，而且這樣的賦值方法是武斷的，我們可以使用任何有序的數字來表達這些選項，舉例來說，你也可以用5來代表非常不同意，用1來代表非常同意。

選擇不同的數值來代表類別將會影響你如何解讀你的分析結果，但只要保持數值有序，你可以選用任何你想要的集合。

最常見的賦值方法是以0或1為起始點。

就像名目型資料，順序型資料可數並可以計算其百分比，但對於順序型資料能否計算百分比仍存在有不同看法。反對的一方認為，就算將類別賦值，依然沒有數學上的實質意義，每個數值只是代表一個特定的類別，而沒有計數的功能。

支持的一方認為，如果相鄰類別間的差距近似相等(例如"非常不同意"與"不同意"的差距近似於"不同意"與"中立")，而且使用相鄰的數值來代表這些類別，則可以用平均數來解釋對類別的賦值。

有些領域非常不鼓勵對順序型資料進行運算，但在有些領域卻是很常見的做法，在使用前務必了解參考你領域中其他人的用法。

區間型

序數型資料介紹到這邊，讓我們回到商店來，你已經在排隊的隊伍中等待了一會兒，看了看你的手錶，你從11:15開始排隊，而現在已經11:30。一天中的時間屬於區間型資料(interval data)，區間型資料因其相鄰點間的區間相等而得名，以時間為例，每一分鐘為六十秒，11:15至11:30的間隔等於12:00至12:15的間隔。

區間資料是具數值屬性的，你可以對它做數學運算，但它並不具有有意義的"0"，換句話說，數值為0並不代表你所量測的事物不存在，例如0:00並不表示時間不存在，它只是代表一天的開始。其他在生活中常見的例子是紀年與溫度，西元0年並不代表時間在此刻之前不存在，溫度0度(華氏或攝氏)也不代表熱不存在。

比率型

時間已經來到11:30，你想著，我已經排了15分鐘的隊了???當你開始這樣想時，你想到的正是比率型資料。比率型資料亦是具數值屬性，除了比率型資料中"0"有意義之外，大致上與區間型資料十分雷同，比率型資料中的"0"表示量測之物不存在，例如0分鐘、0個人排隊、購物籃裡有0件乳製品，上述例子中，"0"確切的表達沒有任何指涉之物的存在，這跟我們先前討論過的區間型資料是不同的，另外，我們常碰到的比率型資料還有身高、體重、年齡、錢等。

區間型資料與比率型資料可以是離散或連續的，離散意思是所測量之物只能有特定數量(通常是整數)，沒有任何可能值介於這些數量間，例如排隊的人數，並不會有1/3個人這樣的數值出現，但你可以計算平均值，意思是我們可以說每行隊伍平均排了4.25個人，但實際人數仍會是一個整數。連續則是表示資料可以是區間內的任何值，你可以買1.25磅的起司或排隊7.75分鐘，請注意這並不表示任何數字都是可能的值，只有特定區間內的數字才能成為可能的值，例如你無法排隊負數分鐘或買負數磅的起司，但這並不影響它的連續性。

為了表達的方便，我們經常將連續的資料四捨五入(或其他捨入方法)，但這並不影響資料的連續性，資料仍然連續，不是離散。

複習一下，在商店的收據上，你能分辨出收據上的資訊是屬於上述的哪一種測量尺度嗎?

日期: 06/01/2014 時間: 11:32am
項目 | 區域 | 道 | 數量 | 成本 (US$)
橘子—磅 | 農產品 | 4 | 2 | 2.58
蘋果—磅 | 農產品 | 4 | 1 | 1.29
乾酪—磅 | 乳製品 | 7 | 1 | 3.49
脫脂牛奶—加侖 | 乳製品 | 8 | 1 | 4.29
豌豆—袋 | 冷凍食品 | 15 | 1 | 0.99
四季豆—袋 | 冷凍食品 | 15 | 3 | 1.77
番茄 | 罐頭 | 2 | 4 | 3.92
馬鈴薯 | 罐頭 | 3 | 2 | 2.38
蘑菇 | 罐頭 | 2 | 5 | 2.95

變數類型與資料類型

如果在網路或書上找尋"資料"的相關資料，經常可以發現變量被描述為上述的一種資料類型，請注意，許多變量並不只只屬於一種類型，反而常因為蒐集資料方法的不同，而屬於不同類型。

以年齡為例，年齡經常被以比率型資料蒐集，但也可以以順序型資料蒐集，例如在問卷中"請問您屬於以下哪個年齡層?"，如此一來，我們就不會有每個受試者的年齡資料，只會知道有多少受試者屬於18-24、25-34或其他年齡層。你可能會為了一個健康研究蒐集參與者確切的膽固醇值，或者只單純的問受試者膽固醇值是否偏高，這樣的一個變量便擁有兩種不同的資料蒐集方式及資料類型。

一個通用的法則是，你可以由含有較多細節的測量尺度轉換為較少細節的測量尺度，反之則不可行。在蒐集資料時，你如果可以蒐集區間型或比率型資料，那必定可以蒐集名目型或順序型資料，但是如果資料天生屬於名目型資料，就如雜貨店中的區域，它不能被以順序型、區間型或比率型資料蒐集，然而很多被以順序型資料蒐集的變量，都也能以區間型或比率型資料蒐集。

順序型資料型態 | 對應的區間型/比率型資料蒐集方法 | 範例
名次 | 紀錄用來決定名次的資料 | 記錄馬拉松跑者的時間，而非記錄其名次
分群尺度 | 直接紀錄資料本身 | 記錄年齡，而不是其分組
替代尺度 | 紀錄替代前的資料 | 紀錄分數，而不是級分

這樣的法則也適用於你在分析或視覺化所擁有的資料時使用，被以比率型資料蒐集的變量，必可以依照專案需求將資料分群，但如果資料是以低階的尺度被蒐集，則在不另外蒐集資料的前提下，將不能提供更高階尺度的資訊，舉例來說，如果以順序型資料來蒐集年齡，未來將無法計算平均年齡，且在視覺化上將只能以年齡級距來表現，不可能將資料以連續資料來表現。

在不對資料蒐集造成過度負擔的情況下，請務必以將來有可能使用的最高階測量尺度來蒐集資料，在處理資料時才發現蒐集的資料並不能滿足視覺化的需求，將是相當令人失望的事！

遺珠之憾

還有許多常被提到的資料型態，因為其意義還存在許多爭議，我們將不在此書中討論，但仍請稍微留意其定義，在其他學習資源中仍有可能看見這些詞彙。

類別資料

前述的名目型及順序型資料都是將資料分為許多類別，有些文獻中將此兩型資料歸為類別資料，其中，名目型資料又稱無序類別資料，順序型資料又稱有序類別資料，又有些文獻認為只有名目型資料為類別資料，且兩者為同義詞，可交互使用，在本書中，我們不另外使用類別資料，且將名目型資料及順序型資料視為兩個不相干的資料型態。

定性與定量資料

定性資料簡單來說就是非數值資料，而定量資料通常是數值資料，也因此可被量化，在此兩種專有名詞上，學者有共識的認為定性資料需要被預先處理，而定量資料須以不同的方法來分析，如直接的觀察紀錄與訪談的腳本。區間型與比率型資料通常是數值資料，因此被普遍地認為是定量資料，而有些歧見發生在名目型及順序型資料上，有人因為它們的類別是被定義的且其並不是真正的數值，而認為它們屬於定性資料，反之，有些人則因為它們仍可數且可計算其百分比而認為它們是定量資料。

為了避免混淆，除了我們將在問卷設計的章節討論到長期形成的定性數據外，本書將通篇使用前述的四種量測尺度，最後，當你未來遇到剛提到的幾個專有名詞時，請務必清楚確認其定義！

</div>

About Data Aggregation

Chapter 2

About Data Aggregation

By Alistair Croll

When trying to turn data into information, the data you start with matter a lot. Data can be simple factoids—of which someone else has done all of the analysis—or raw transactions, where the exploration is left entirely to the user.

Level of Aggregation | Number of metrics | Description
Factoid | Maximum context | Single data point; No drill-down
Series | One metric, across an axis | Can compare rate of change
Multiseries | Several metrics, common axis | Can compare rate of change, correlation between metrics
Summable multiseries | Several metrics, common axis | Can compare rate of change, correlation between metrics; Can compare percentages to whole
Summary records | One record for each item in a series; Metrics in other series have been aggregated somehow | Items can be compared
Individual transactions | One record per instance | No aggregation or combination; Maximum drill-down

Most datasets fall somewhere in the middle of these levels of aggregation. If we know what kind of data we have to begin with, we can greatly simplify the task of correctly visualizing them the first time around.

Let’s look at these types of aggregation one by one, using the example of coffee consumption. Let’s assume a café tracks each cup of coffee sold and records two pieces of information about the sale: the gender of the buyer and the kind of coffee (regular, decaf, or mocha).

The basic table of these data, by year, looks like thisThese are completely made up coffee data, BTW.:

Year | 2000 | 2001 | 2002 | 2003 | 2004 | 2005 | 2006 | 2007 | 2008
Total sales | 19,795 | 23,005 | 31,711 | 40,728 | 50,440 | 60,953 | 74,143 | 93,321 | 120,312
Male | 12,534 | 16,452 | 19,362 | 24,726 | 28,567 | 31,110 | 39,001 | 48,710 | 61,291
Female | 7,261 | 6,553 | 12,349 | 16,002 | 21,873 | 29,843 | 35,142 | 44,611 | 59,021
Regular | 9,929 | 14,021 | 17,364 | 20,035 | 27,854 | 34,201 | 36,472 | 52,012 | 60,362
Decaf | 6,744 | 6,833 | 10,201 | 13,462 | 17,033 | 19,921 | 21,094 | 23,716 | 38,657
Mocha | 3,122 | 2,151 | 4,146 | 7,231 | 5,553 | 6,831 | 16,577 | 17,593 | 21,293

Factoid

A factoid is a piece of trivia. It is calculated from source data, but chosen to emphasize a particular point.

Example: 36.7% of coffee in 2000 was consumed by women.

Series

This is one type of information (the dependent variable) compared to another (the independent variable). Often, the independent variable is time.

Year | 2000 | 2001 | 2002 | 2003
Total sales | 19,795 | 23,005 | 31,711 | 40,728

In this example, the total sales of coffee depends on the year. That is, the year is independent (“pick a year, any year”) and the sales is dependent (“based on that year, the consumption is 23,005 cups”).

A series can also be some other set of continuous data, such as temperature. Consider this table that shows how long it takes for an adult to sustain a first-degree burn from hot water. Here, water temperature is the independent variableUS Government Memorandum, Consumer Product Safety Commission, Peter L. Armstrong, Sept. 15, 1978.:

Water Temp °C (°F) | Time for 1st Degree Burn
46.7 (116) | 35 minutes
50 (122) | 1 minute
55 (131) | 5 seconds
60 (140) | 2 seconds
65 (149) | 1 second
67.8 (154) | Instantaneous

And it can be a series of non-contiguous, but related, information in a category, such as major car brands, types of dog, vegetables, or the mass of planets in the solar systemNational Space Science Data Center, NASA: http://nssdc.gsfc.nasa.gov/planetary/factsheet/planet_table_ratio.html:

Planet | Mass relative to earth
Mercury | 0.0553
Venus | 0.815
Earth | 1
Mars | 0.107
Jupiter | 317.8
Saturn | 95.2
Uranus | 14.5
Neptune | 17.1

In many cases, series data have one and only one dependent variable for each independent variable. In other words, there is only one number for coffee consumption for each year on record. This is usually displayed as a bar, time series, or column graph.

In cases where there are several dependent variables for each independent one, we often show the information as a scatterplot or heat map, or do some kind of processing (such as an average) to simplify what’s shown. We’ll come back to this in the section below, Using visualization to reveal underlying variance.

Multiseries

A multiseries dataset has several pieces of dependent information and one piece of independent information. Here are the data about exposure to hot water from before, with additional dataUS Government Memorandum, Consumer Product Safety Commission, Peter L. Armstrong, Sept. 15, 1978.:

Water Temp °C (°F) | Time for 1st Degree Burn | Time for 2nd & 3rd Degree Burns
46.7 (116) | 35 minutes | 45 minutes
50 (122) | 1 minute | 5 minutes
55 (131) | 5 seconds | 25 seconds
60 (140) | 2 seconds | 5 seconds
65 (149) | 1 second | 2 seconds
67.8 (154) | Instantaneous | 1 second

Or, returning to our coffee example, we might have several series:

Year | 2000 | 2001 | 2002 | 2003 | 2004 | 2005
Male | 12,534 | 16,452 | 19,362 | 24,726 | 28,567 | 31,110
Regular | 9,929 | 14,021 | 17,364 | 20,035 | 27,854 | 34,201

With this dataset, we know several things about 2001. We know that 16,452 cups were served to men and that 14,021 cups served were regular coffee (with caffeine, cream or milk, and sugar).

We don’t, however, know how to combine these in useful ways: they aren’t related. We can’t tell what percentage of regular coffee was sold to men or how many cups were served to women.

In other words, multiseries data are simply several series on one chart or table. We can show them together, but we can’t meaningfully stack or combine them.

Summable Multiseries

As the name suggests, a summable multiseries is a particular statistic (gender, type of coffee) segmented into subgroups.

Year | 2000 | 2001 | 2002 | 2003 | 2004 | 2005 | 2006 | 2007 | 2008
Male | 12534 | 16452 | 19362 | 24726 | 28567 | 31110 | 39001 | 48710 | 61291
Female | 7261 | 6553 | 12349 | 16002 | 21873 | 29843 | 35142 | 44611 | 59021

Because we know a coffee drinker is either male or female, we can add these together to make broader observations about total consumption. For one thing, we can display percentages.

Additionally, we can stack segments to reveal a whole:

One challenge with summable multiseries data is knowing which series go together. Consider the following:

Year | 2000 | 2001 | 2002 | 2003 | 2004
Male | 12534 | 16452 | 19362 | 24726 | 28567
Female | 7261 | 6553 | 12349 | 16002 | 21873
Regular | 9929 | 14021 | 17364 | 20035 | 27854
Decaf | 6744 | 6833 | 10201 | 13462 | 17033
Mocha | 3122 | 2151 | 4146 | 7231 | 5553

There is nothing inherent in these data that tells us how we can combine information. It takes human understanding of data categories to know that Male + Female = a complete set and Regular + Decaf + Mocha = a complete set. Without this knowledge, we can’t combine the data, or worse, we might combine it incorrectly.

It’s Hard to Explore Summarized Data

Even if we know the meaning of these data and realize they are two separate multiseries tables (one on gender and one on coffee type) we can’t explore them deeply. For example, we can’t find out how many women drank regular coffee in 2000.

This is a common (and important) mistake. Many people are tempted to say:

36.7% of cups sold in 2000 were sold to women.

And there were 9,929 cups of regular sold in 2000.

Therefore, 3,642.5 cups of regular were sold to women.

But this is wrong. This type of inference can only be made when you know that one category (coffee type) is evenly distributed across another (gender). The fact that the result isn’t even a whole number reminds us not to do this, as nobody was served a half cup.

The only way to truly explore the data and ask new questions (such as “How many cups of regular were sold to women in 2000?”) is to have the raw data. And then it’s a matter of knowing how to aggregate them appropriately.

Summary Records

The following table of summary records looks like the kind of data a point-of-sale system at a café might generate. It includes a column of categorical information (gender, where there are two possible types) and subtotals for each type of coffee. It also includes the totals by the cup for those types.

Name | Gender | Regular | Decaf | Mocha | Total
Bob Smith | M | 2 | 3 | 1 | 6
Jane Doe | F | 4 | 0 | 0 | 4
Dale Cooper | M | 1 | 2 | 4 | 7
Mary Brewer | F | 3 | 1 | 0 | 4
Betty Kona | F | 1 | 0 | 0 | 1
John Java | M | 2 | 1 | 3 | 6
Bill Bean | M | 3 | 1 | 0 | 4
Jake Beatnik | M | 0 | 0 | 1 | 1
Totals | 5M, 3F | 16 | 8 | 9 | 33

This kind of table is familiar to anyone who’s done basic exploration in a tool like Excel. We can do subcalculations:

There are 5 male drinkers and 3 female drinkers

There were 16 regulars, 8 decafs, and 9 mochas

We sold a total of 33 cups

But more importantly, we can combine categories of data to ask more exploratory questions. For example: Do women prefer a certain kind of coffee? This is the kind of thing Excel, well, excels at, and it’s often done using a tool called a Pivot Table.

Here’s a table looking at the average number of regular, decaf, and mocha cups consumed by male and female patrons:

Row Labels | Average of Regular | Average of Decaf | Average of Mocha
F | 2.67 | 0.33 | 0.00
M | 2.00 | 1.75 | 2.00
Grand Total | 2.29 | 1.14 | 1.14

Looking at this table, we can see a pretty clear trend: Women like regular; men seem evenly split across all three types of coffeeThere aren’t enough data to make a statistically reliable statement like this. But this is all made-up data anyway, so stop thinking so much about coffee consumption..

The thing about these data, however, is they have still been aggregated somehow. We summarized the data along several dimensions—gender and coffee type—by aggregating them by the name of the patron. While this isn’t the raw data, it’s close.

One good thing about this summarization is that it keeps the dataset fairly small. It also suggests ways in which the data might be explored. It is pretty common to find survey data that looks like this: for example, a Google Form might output this kind of data from a survey that says:

Producing the following data in the Google spreadsheet:

Timestamp | What is your name? | Gender? | Regular | Decaf | Mocha
1/17/2014 11:12:47 | Bob Smith | Male |  | 4 | 3

Using Visualization to Reveal Underlying Variance

When you have summary records or raw data, it’s common to aggregate in order to display them easily. By showing the total coffee consumed (summing up the raw information) or the average number of cups per patron (the mean of the raw information) we make the data easier to understand.

Consider the following transactions:

Name | Regular | Decaf | Mocha
Bob Smith | 2 | 3 | 1
Jane Doe | 4 | 0 | 0
Dale Cooper | 1 | 2 | 4
Mary Brewer | 3 | 1 | 0
Betty Kona | 1 | 0 | 0
John Java | 2 | 1 | 3
Bill Bean | 3 | 1 | 0
Jake Beatnik | 0 | 0 | 1
Totals | 16 | 8 | 9
Averages | 2 | 1 | 1.125

We can show the average of each coffee type consumed by cup as a summary graphic:

But averages hide things. Perhaps some people have a single cup of a particular type, and others have many. There are ways to visualize the spread, or variance, of data that indicate the underlying shape of the information, including heat charts, histograms, and scatterplots. When keeping the underlying data, you can wind up with more than one dependent variable for each independent variable.

A better visualization (such as a histogram, which counts how many people fit into each bucket or range of values that made up an average) might reveal that a few people are drinking a lot of coffee, and a large number of people are drinking a small amount.

Consider this histogram of the number of cups per patron. All we did was tally up how many people had one cup, how many had two, how many had three, and so on. Then we plotted how frequently each number occurred, which is why this is called a frequency histogram.

The average number of cups in this dataset is roughly 3. And the mode, or most common number, is 2 cups. But as the histogram shows, there are three heavy coffee drinkers who’ve each consumed 7 cups, pushing up the average.

In other words, when you have raw data, you can see the exceptions and outliers, and tell a more accurate story.

Even these data, verbose and informative as they are, aren’t the raw information: they’re still aggregated.

Aggregation happens in many ways. For example, a restaurant receipt usually aggregates orders by table. There’s no way to find out what an individual person at the table had for dinner, just the food that was served and what it cost. To get to really specific exploration, however, we need data at the transaction level.

Individual Transactions

Transactional records capture things about a specific event. There’s no aggregation of the data along any dimension like someone’s name (though their name may be captured). It’s not rolled up over time; it’s instantaneous.

Timestamp | Name | Gender | Coffee
17:00 | Bob Smith | M | Regular
17:01 | Jane Doe | F | Regular
17:02 | Dale Cooper | M | Mocha
17:03 | Mary Brewer | F | Decaf
17:04 | Betty Kona | F | Regular
17:05 | John Java | M | Regular
17:06 | Bill Bean | M | Regular
17:07 | Jake Beatnik | M | Mocha
17:08 | Bob Smith | M | Regular
17:09 | Jane Doe | F | Regular
17:10 | Dale Cooper | M | Mocha
17:11 | Mary Brewer | F | Regular
17:12 | John Java | M | Decaf
17:13 | Bill Bean | M | Regular

These transactions can be aggregated by any column. They can be cross-referenced by those columns. The timestamps can also be aggregated into buckets (hourly, daily, or annually). Ultimately, the initial dataset we saw of coffee consumption per year results from these raw data, although summarized significantly.

Deciding How to Aggregate

When we roll up data into buckets, or transform it somehow, we take away the raw history. For example, when we turned raw transactions into annual totals:

We anonymized the data by removing the names of patrons when we aggregated it.

We bucketed timestamps, summarizing by year.

Either of these pieces of data could have shown us that someone was a heavy coffee drinker (based on total coffee consumed by one person, or based on the rate of consumption from timestamps). While we might not think about the implications of our data on coffee consumption, what if the data pertained instead to alcohol consumption? Would we have a moral obligation to warn someone if we saw that a particular person habitually drank a lot of alcohol? What if this person killed someone while driving drunk? Are data about alcohol consumption subject to legal discovery in a way that data about coffee consumption needn’t be? Are we allowed to aggregate some kinds of data but not others?

Can we address the inherent biases that result from choosing how we aggregate data before presenting it?

The big data movement is going to address some of this. Once, it was too computationally intensive to store all the raw transactions. We had to decide how to aggregate things at the moment of collection, and throw out the raw information. But advances in storage efficiency, parallel processing, and cloud computing are making on-the-fly aggregation of massive datasets a reality, which should overcome some amount of aggregation bias.

</div> </div>

Collecting Data

資料蒐集

蒐集資料就如同為食譜備齊材料一般。若想要料理出一道令人驚豔的菜餚，首先要從原材料著手。所以，這就表示在料理之前，你將會面臨一大堆繁瑣的抉擇。

例如，如果你需要使用蜂蜜，你會選用一般的蜂蜜？還是會指定特定品種，如桂花、龍眼等的蜂蜜？還有品牌的偏好？這蜂蜜是生蜜還是熟成蜜，要有機的嗎？要跟在地小農購買呢，還是到超市買呢？還有還有，怎樣去買？沒空要找人幫忙跑腿？會不會買了不是要用的蜂蜜啊？如果這樣要不乾脆網購算了？還有還有……

同樣的，在蒐集資料之前，你也會需要作出林林總總的決定。要用紙本問卷隨機讓路人填寫開放式問題嗎？欸，好像看起來沒什麼用處啊！？不同種類的資料能在不同場地以不同方法取得的話自然是極好的，可萬一你沒錢沒空呢？我難過…

在本章節内，我們會談到資料的搜集方法能夠幫助你在最佳位置找出最需要的資訊，以及如何篩選出真正需要並使用到的資訊。地方的媽媽們，給開司一罐啤酒…

Intro to Survey Design

Chapter 3

Intro to Survey Design

By Ginette Law

Most people think conducting a survey is as simple as writing a bunch of questions and asking people to answer them. Easy, right? Well, sort of: it’s a little more complicated than that if you want to collect the best data possible. Don’t worry, though: we’re about to go over some key elements of survey design, starting with the purpose of your survey.

Purpose of a Survey

The first step in designing a good survey is to identify its purpose before you create it. A good survey collects accurate and verifiable data that allow you to make concrete claims. This is easier to do if you have a clear purpose to guide you when deciding what information you want to collect your respondents.

A survey generally helps you do one or more of the following:

describe something;

describe how things are related;

explain a relationship; or

influence something

So how do you come up with a nice, well-defined purpose for your survey? Try asking yourself the following questions:

What are you trying to achieve with your survey?

What, precisely, do you want to know?

Why is this important to know?

Is there other information that could be useful? Why?

Is conducting a survey the right method for the type of data you’re collecting?

To see this process in action, let’s say you’re a media tycoon trying to increase your profits from television subscribers, so you’re interested in the potential returns from expanding to an internet market. You decide the main question and purpose of your survey are:

Research Question
Purpose

What percentage of television viewers watch their favorite television shows online?
Describe a variable

The percentage of people who watch television programs online.

After thinking about it, you realize that it would be helpful to get more details, so you ask yourself some of those questions we just suggested:

Question | Answer
What are you trying to achieve with your survey? | To evaluate the profit potential of the internet market.
What do you want to know precisely? | How many people currently or would like to be able to stream shows online.
Why is this important to know? | It will help determine where we can improve our online service and whether it’s worth the investment.
Is there other information that could be useful? Why? | Which age group is more likely to watch TV online.
Is conducting a survey the right method for the type of data you’re collecting? | Yes.

Based on your answers to these questions, you expand the scope of your survey slightly:

Research Question
Purpose

How can we maximize profits from online viewership?
Influence something:

Find out what needs to be done to improve our online service.

Do younger television users tend to stream more television programs online than older viewers?
Describe how things are related:

Describe how a viewer’s age is related to how much TV he or she watches online.

If one group watches more programs online, why?
Explain a relationship:

Explain why one age group prefers watching TV online more than another age group does.

Now you have a nice set of clearly-defined questions that your survey should address. The next step is to choose the best type of survey to help you expand your business.

Data and numbers can be very powerful, but sometimes people conduct surveys for the wrong reasons. You will miss the opportunity to gain meaningful insight with your survey if:

You don’t really care about the results; you just want to show people you’ve got numbers.

You misuse the data collected.

You are more concerned by how people will receive your findings than you are with having reliable results.

You have already determined what the results “should” be.

Remember, we do research to gain insight and to test hypotheses. That means it’s important to try and collect the most accurate and representative data possible and not just the data that support your own preconceived ideas and biases.

Types of Surveys

Now that you’ve identified the purpose of your survey, you can use this to help you choose a type of survey. When it comes to choosing a survey type, the first big decision you’ll have to make is how you want to distribute your survey to respondents. Surveys can be self-administered or administered.

Self-Administered and Administered Surveys

Self-administered simply means that respondents fill out a questionnaire by themselves, whereas administered means that an interviewer asks the questions. Each of these methods has advantages and disadvantages, summarized below.

Self-Administered Surveys

Pros
Cons

Good for limited budgets

Large geographic reach

Respondents can answer when convenient and at their own pace

More difficult to distribute to populations with limited literacy

Risk of respondents misinterpreting questions

Risk of lower completion rate

Requires extra effort to verify that the respondent was actually the person to whom the survey was addressed

Administered Surveys

Pros
Cons

Tighter quality control

Interviewers can clarify questions for respondents

Can collect richer and more in-depth information

Easier to reach specific or marginalized populations (e.g., the elderly or the homeless)

Can be expensive

Can be time consuming

Risk of interviewer effect (e.g., an interviewer’s interpretation of a question could bias some of the results)

The table above mentions the interviewer effect. This is a phenomenon whereby the interviewers themselves influence the responses. This could happen because of the way they ask questions or phrase explanations, or because of another social factor altogether. For example, maybe the interviewer is incredibly good-looking and the respondent unintentionally gives answers that are biased towards impressing the interviewer!

Self-administered and administered surveys are each associated with certain types of surveys, as shown below:

Types of Self-Administered Surveys

Web and Email Surveys

One of the easiest and quickest ways you can conduct a survey is online. You can either have people fill out questionnaires directly on a website or send them a questionnaire via e-mail. There are even survey sites that let you design questionnaire forms, collect data, and analyze responses in real-time at an affordable rate. You’ll find links to a few of these sites in the resources section.

Don’t forget to read the Privacy Policies and Terms of Use before signing up though—especially if you’re dealing with sensitive data—because these services are often subject to the laws of the countries in which they’re based.

The downside to all of this convenience is that only regular internet or computer users are likely to fill out your survey.

Web and email

Pros
Cons

Fairly fast results

Integrated support tools for images, sounds, video

Relatively cheap

Limited to people who have internet access

Respondents can be a self-selected group of regular computer-users

Web

Pros
Cons

Can invite participants through social media

Can use survey sites

Survey sites are subject to the laws of the countries in which they’re based

Email

Pros
Cons

Easier to survey a specific group

Need email addresses in order to distribute survey

Mail Surveys

Mail surveys are much like email surveys except that you’re sending a paper version of the questionnaire by mail. They aren’t limited to the population of internet users, but they can be more expensive because you have to pay for printing, paper, envelopes, and postage. You may also end up with a lower response rate because some people think it’s a pain to mail anything, not just your survey. Including a pre-addressed, stamped envelope with the survey will often improve your overall response rate, but will, again, increase the overall cost to you.

Mail

Pros
Cons

Open to a wider population

More expensive

Lower response rate

Types of Administered Surveys

Phone Surveys

Compared to online surveys, phone surveys may take less time for you to collect all the data you need because respondents answer questions immediately. However, if you need to complete a certain number of surveys and most people hang up on you, it might take longer to collect data by phone!

It helps if questions are short and clear so that respondents can easily understand them over the phone. Try also to not take more than 15 to 20 minutes of your respondents’ time (both to be respectful of their time and to avoid losing their attention).

Another thing to keep in mind is that the interviewer and the respondent can’t see each other. This can be both good and bad: good because it lessens the interviewer effect but bad because both parties may miss non-verbal cues.

Pros
Cons

Relatively fast results

Higher quality control because trained interviewers administer the survey

Can be more expensive

Can’t use images or video when asking questions

Need a list of reliable phone numbers

Face-to-Face Surveys

Unlike phone surveys, face-to-face surveys allow the interviewer and the respondent to see each other’s facial expressions and body language. This can be helpful because the additional physical cues can help the interviewer and the respondent understand better understand each other; however, it can also lead to the respondent being further influenced by the interviewer’s behavior and appearance.

Face-to-face surveys are limited geographically if interviewers and respondents have to meet in person. If they don’t have to meet in person, the surveys can be conducted online using video conferencing software such as Skype or Google Hangouts.

Pros
Cons

Can collect more in-depth information

Good for specific or hard-to-reach populations

Better for long surveys

Can be more expensive

May have a stronger interviewer effect

Can take longer to train interviewers and to complete enough surveys

Handouts

So far, we’ve talked about self-administered and administered surveys, but one of the most frequently-encountered type of survey is actually a combination of these. Say you want to hand out paper surveys and have people complete and return them immediately. The survey itself is self-administered, but since you have a trained person there who is available to answer questions, it also has features of an administered survey.

For this reason, handouts can be a good option if you have a limited budget but want to make sure you have someone available when the survey is completed to clarify any questions your respondents might have.

One of the disadvantages of handouts is that people may be rushed to complete the survey if you are catching them in passing, which can affect the quality of your data. You will also be limited to the population that is physically present in the location where you are giving the survey. This may not be an issue if you are targeting a specific group, such as college students, shoppers at a particular store, or residents of a certain region. If you’re looking for a more general audience, however, you may consider handing the survey out in several different locations to reach a more diverse audience.

Choosing a Survey Type

Whew! That is a lot of information so far. As you can see, there are many types of surveys, each with its own pros and cons. With so many factors to consider, how do you decide which one to choose? Let’s walk through a few scenarios of choosing a survey type. Remember, you’re a media tycoon trying to expand your media empire.

Let’s say you want to evaluate the potential of the internet market. If you’re only interested in the online viewing habits of internet users, then a web or email survey cheaply and conveniently targets the group you’re interested in. On the other hand, if you want to know how much of the general television viewing audience watches shows online, but you don’t want to spend the time or the money to train interviewers, then a mail survey is a better option.

But wait a minute: you’re a media tycoon! You have lots of money and you’re a total data geek. You want to collect more in-depth information like:

How do families watch television programs in their homes?

Which family members tend to watch TV online?

What type of television program does each family member like to watch?

Do they watch programs together?

If so, what types of programs do they watch together? Do they ever fight about what to watch?

In this case, a face-to-face survey is a great option because you can interview all the members of a household at the same time and collect higher quality data about the dynamics of television viewing in families.

Now let’s say you’ve analyzed the data and the numbers look promising, so you decide to create a new online viewing experience for your users. You want to emphasize how great and cutting-edge your new service is by creating a catchy name and logo for it. If you want to see what people think about different logos, then phone surveys won’t be helpful. On the other hand, if you want to see what people think about different names and slogans, then phone surveys are fine. If your new service is specifically targeting a younger audience of avid internet users, then you can avoid the extra cost of phone surveys and use an online survey instead.

If you have the time and resources to do so, you can also consider using more than one type of survey. For example, you may want to do an online and phone version of the same survey to increase response rates and improve your sample in both size and diversity. Or you may start by doing an online survey and then conduct a face-to-face survey afterwards to gain more insight from your initial results.

We hope these scenarios show you that choosing a survey type isn’t a one-size-fits-all process. There isn’t a nice and tidy formula that you can use to pick the perfect survey. Instead, it’s up to your good judgment to balance the resources you have with the goals you’re trying to achieve.

</div>

Types of Survey Questions

Chapter 4

Types of Survey Questions

By Ginette Law

After you’ve decided what type of survey you’re going to use, you need to figure out what kinds of questions to include. The type of question determines what level of data can be collected, which in turn affects what you can do with the data later on. For example, if you ask about age and you record it as a number (e.g., 21, 34, 42), you’ll have numeric data that you can perform mathematical operations on. If you instead ask for a person’s age group (e.g., 18-24, 25-34, 35-44), you’ll have ordered categorical data: you’ll be able to count how many people were in each group, but you won’t be able to find the mean age of your respondents. Before you create your survey, it’s important to consider how you want to analyze your final results so you can pick question types that will give you the right kind of data for those analyses.

Open vs. Closed Questions

Open (or open-ended) questions are used when you want to collect long-form qualitative data. You ask the person a question without offering any specific answers from which to choose. This is often a good option to discover new ideas you may have overlooked or did not know existed. In a closed question, however, a limited number of specific responses are offered to answer the question. Closed questions are used when you already have an idea what categories your answers will fall into or you’re only interested in the frequency of particular answers.

Let’s pretend you work for an ice cream parlour called Fictionals. You want to know why some people don’t come to your store. Here are open and closed questions you could ask to find out their reasons.

In the closed question, you offer the most probable reasons why people would not go eat ice cream at your store, whereas in the open question, you just ask why they don’t eat at Fictionals and allow respondents to answer however they want.

When you ask an open-ended question you gain new insight to why some people might not try your ice cream: it’s not because they don’t like ice cream or they don’t like Fictionals, it’s because they can’t eat it! With this new insight, you could choose to offer a new type of ice cream and attract a clientele you didn’t appeal to before.

This is a distinct advantage of open-ended questions, and it’s particularly useful if you’re doing exploratory research and will primarily be reading through the responses to get a sense of how people responded. However, it takes considerably more work to analyze and summarize these data.

When you think that most people will give more or less the same answer, you can combine an open and closed question to get the best results. This is usually done by offering a number of predefined choices, but leaving room for participants to explain their response if it falls under the “Other” category. Here is how the same question would look if you transformed it into a combined open and closed question:

Closed Question Types

Multiple Choice Questions

Sometimes you’ll want the respondent to select only one of several answers. If your survey is online, you use radio buttons (or circles), which indicate to choose just one item. If you want to allow the respondent to choose more than one option, you should use a checkbox instead. Here is what it looks like in practice:

If your survey is a paper or administered survey, you would use printed or verbal directions to indicate how many options the respondent should pick (e.g., “Please choose one”; “Please select up to 3”; “Please select all that apply”).

Answer choices should not overlap and should cover all possible options. This is called having “mutually exclusive and exhaustive” categories and is crucial for sound survey design. Consider the situation below:

In the left set of choices, someone who is 25 wouldn’t know whether to select the 18-25 button or the 25-35 button, and someone who is 60 wouldn’t have any option to select at all! The set of age categories on the right is better because it includes all possible age groups, the age choices don’t overlap, and it allows people the choice to withhold that information if they choose to.

It is advisable to include a “Prefer not to answer” option for questions that may be of a personal nature, such as race, ethnicity, income, political affiliation, etc. Without this option, people may skip the question altogether, but you won't be able to tell later on who skipped the question on purpose and who missed it accidentally, which can be an important difference when you analyze your data.

It won’t always be practical to list every possible option. In that case, including “None of the above” or “Other” choices may be helpful. This makes it less likely that people will just skip the question and leave you with missing data. You may also consider including a “Not Applicable” (i.e., “N/A”) choice for questions that may not apply to all of your respondents, or a “Don’t know” choice for questions where a respondent may not not have the information available.

Dichotomous Questions

Dichotomous questions are a specific type of multiple choice question used when there are only two possible answers to a question. Here are some classic examples of dichotomous type questions:

You can use dichotomous questions to determine if a respondent is suited to respond to the next question. This is called a filter or contingency question. Filters can quickly become complex, so try not to use more than two or three filter questions in a row. Here’s an example of the logic behind a filter question:

In an online survey or administered survey, answering “Yes” to the question “Do you ever eat ice cream at Fictionals Ice Cream Parlour?” would take a respondent to the question about how often they do, while selecting “No” would open the checkbox response question asking why they don’t eat there. In a paper survey, all questions would need to be listed on the page and couldn’t be presented conditionally based on a specific response. You could use a flowchart form like in the diagram above, or you could employ skip directions, as illustrated below.

Scaled Questions

Scaled questions are used to measure people’s attitudes, opinions, lifestyles and environments. There are many different types but the most commonly used are Likert and slider scales. The main difference between the two is that a Likert scale consists of ordered categories whereas a slider scale allows respondents to make a mark indicating their response anywhere along a scale. As a result, a Likert scale-type question will give you data that are measured at an ordinal level while a slider scale will give you data that are measured at an interval level.

In the example above, the same question is asked twice, once with a Likert scale and once with a sliding scale. Notice that in the Likert scale there are five categories. Scales with an odd number of categories allow participants to agree, disagree, or indicate their neutrality; scales with an even number of categories only allow participants to agree or disagree. If you want to allow for neutrality, scales with an odd number of categories are usually preferred; however, you should use an even number if you want to require the respondent to choose one direction or the other. This is called a “forced question.”

Likert Scale

Likert scales are usually limited to a five-category scale, as it is often difficult to clearly display a larger number of categories. The most common Likert scales you will find are:

Opinions and attitudes: “How much do you agree with…?”
Possible answers: strongly agree, agree, neither agree or disagree, disagree, strongly disagree

Frequency: “How often do you…?”
Possible answers: always, often, sometimes, rarely, never

Quality: “In general, how do you rate the quality of…?”
Possible answers: excellent, very good, good, fair, poor

Importance: “How important would you say {topic} is…”
Possible answers: very important, important, somewhat important, not at all important

Slider Scale

Slider scales are useful when you want to have a more precise reading of a respondent’s views. These scales are most easily implemented in an online survey, since the computer will calculate the position of the marker; when used on a paper survey, you will need to have someone measure the marker’s location on the scale manually. Slider scales may be more practical than Likert scales when conducting a survey that is being translated into multiple languages since text categories are not always easy to translate.

When creating a scale, whether Likert or slider, never forget to label the first and last points in your scale. Otherwise, your respondents may misinterpret your scale and give you inaccurate or misleading data.

Question Wording

It’s extremely important to think about how you word your questions when developing a survey. Here are a few key things to keep in mind:

Focus

Each question should be specific and have a defined focus.

How many times have you eaten ice cream in the last month?

This question focuses on one issue (frequency of ice cream eating) and is specific (a certain time period).

Do you avoid eating ice cream because you are on a diet? [Answer: Yes, I avoid ice cream, but not because I am on a diet]

This question is poorly worded for a few reasons: it is both leading and too specific. It assumes that the respondent is in fact on a diet and that this is why he or she doesn’t eat ice cream. A better wording would be “Why do you avoid eating ice cream?” with “I’m dieting” as one of a number of options that the participant can choose from.

You should also avoid using the word “and” if it is connecting two different ideas within a single question. Remember, each question should focus on only one issue at a time: otherwise, you won’t be collecting the best data that you can. By compounding multiple thoughts into a single question, you reduce the accuracy of participants’ responses and thereby limit the claims that you can make from those data. Instead, consider using filter questions to obtain the desired information. For example:

Precision

Not everyone will interpret all words and phrasings in the same way, even if the meaning seems obvious to you. To reduce the chance of participants misinterpreting your question, you can parenthetically define ambiguous terms or give additional context as appropriate. Try also to avoid confusing wordings, such as those that use double negatives or many subordinate clauses. Be careful to also not use words that are loaded or have many highly-emotional connotations.

Do
Don’t

Have you eaten chocolate ice cream in the past month?

Ice cream with peanuts should not be permitted.

Ice cream vendors should not disclaim having a peanut-free shop.

Rocky Road, Mint Chocolate Chip, Dutch Chocolate and Hot Fudge ice creams are Fictionals’ best sellers. Have you eaten any chocolate ice cream in the last month?

Ice cream with peanuts should be banned.

Ice cream vendors should accept to have a peanut-free shop.

Brevity

Your questions should be relatively short, except where additional wording is absolutely necessary to provide context or to clarify terminology. Long, complex questions can quickly become confusing to participants and increase the chances that they will respond without fully understanding what is being asked, or worse, skip the question entirely! If your question seems to be getting too long, consider whether there is any unnecessary information that you can take out or if you can split it up into several smaller questions. Alternately, you may want to have a short paragraph of explanatory text that is separate from the question itself: that way participants will have all necessary background information, but can also easily pick out and digest what is actually being asked.

Do | Don’t
Please list your three favorite ice creams in order of preference. | Can you please tell us what ice cream flavors you like and what are your first, second, and third favorites?

Biased and Leading Questions

Biased or leading questions can easily skew your answers if you do not pay close attention to your wordings. Avoid over-emphasizing your statements and keep an eye out for questions that create “social desirability effects” where respondents may be more likely to answer according to what society views as proper or socially or morally acceptable.

Do | Don’t
Do you agree that ice cream vendors in our city should offer peanut-free products? | Do you agree that ice cream vendors that serve peanuts are a severe hazard for the well-being of our children?

Notice how the second question tries to bias the respondent by using strong or affective phrases such as “severe hazard.” You should try to keep your questions as value-free as you can: if the question itself suggests how the participant is expected to answer, it should be reworded.

A Few Final Things…

There are few final things to consider when developing your survey questions:

If possible, consider varying the type of questions you use to keep your respondents engaged throughout the survey. You have a variety of question types to choose from, so mix it up where you can! (But do keep in mind that each type of question has a specific purpose and gives you more or less specific data.)

Think about how certain words can have different interpretations and remember that meanings are often embedded in culture and language. Be sensitive to different cultures if you are examining several groups within a population.

Designing Your Questionnaire

Once you have identified the purpose of your survey and chosen which type you will use, the next step is to design the questionnaire itself. In this section, we’ll look at the structure, layout and ordering of questions. We’ll also talk about some key things to keep in mind once you’re done.

Questionnaire Structure

Do you remember when you were taught in grade school that every essay should have an introduction, body and conclusion? Questionnaires should also have a certain structure. The major parts of a questionnaire include the introduction, main topic, transitions between topics, demographic questions, and conclusion.

Introduction

Always start your survey with a brief introduction which explains:

the purpose of the survey;

who is conducting it;

the voluntary nature of the participant’s involvement;

the respect for confidentiality; and

the time required to complete the survey.

Main Topics

Generally, it is best to start with general questions and then move on to more specific questions.

Ask about objective facts before asking more subjective questions.

Order questions from the most familiar to least.

Make sure that the answer to one question does not impact how the participant interprets the following question.

Transitions Between Topics

Use transitions between groups of questions to explain a new topic or format. For example: “The next few questions are related to the frequency of your TV viewing habits. Please choose the answer that best describes your situation.”

Demographic Questions

Unless you are using demographic questions as filtering criteria for survey eligibility, it is usually better to put them near the end of a survey. Typical demographic questions include:

gender;

age;

income;

nationality or geographic location;

education; and

race or ethnicity.

Conclusion

Thank the participants for their contribution and explain how it has been valuable to the project. Reiterate that their identities will be kept confidential and that results will be anonymized. If you wish, you can also include your contact information in case they have any additional questions related to the survey and also ask for their contact information if you are offering an incentive for completing the survey.

If you contact respondents in advance to invite them to participate in the survey, you should also explain why they have been selected to participate, when the survey will take place, and how they can access it.

General Layout

There are a few good rules to follow when designing the layout of your survey:

Put your introduction and conclusion on separate pages from your questions.

The format should be easy to follow and understand.

Check that filter questions work as intended.

Stick to questions that meet your goals rather than just asking every question you can think of. In general, survey completion rates tend to diminish as surveys become longer.

Leave enough space for respondents to answer open-ended questions.

Make sure to include all likely answers for closed-ended questions, including an “Other” option in case respondents feel that none of the provided answers suits them. Where appropriate, also include “Not Applicable” and “Choose not to respond” options.

Ensure that questions flow well and follow a logical progression. Begin the survey with more general questions and then follow with more specific or harder issues. Finish with general demographic topics (e.g., age, gender, etc.), unless you are using these to screen for eligibility at the beginning of the survey. Group questions by theme or topic.

Some Further Considerations

Consider Your Audience

Think about the people who will be participating in your survey. Let’s say, for example, that you want to conduct a survey in a school where many students have recently immigrated to the country as refugees with their families and don’t yet speak the local language well.

In what language(s) should you conduct the survey? Should you only conduct the survey in the local language or instead prepare it in several languages? If you’re preparing different versions of the survey, who will ensure consistency among the translations?

What kind of vocabulary should you use? These are students who may not be taking the questionnaire in their native language. You will probably want to use fairly basic language, especially if many are not native speakers. We have not specified their age, but it would be useful to take this information into consideration when thinking about vocabulary.

Word Instructions and Questions Clearly and Specifically

It is important that your instructions and questions are simple and coherent. You want to make sure that everyone understands and interprets the survey in the same way. For example, consider the difference between the following two questions.

When did you first notice symptoms? ______________________

When did you first notice symptoms (e.g., MM/YYYY)? __ __ / __ __ __ __

If you want the question to be open to interpretation so that people can give answers like, “after I came back from my last trip,” then the first option is okay. However, if you’re really wondering how long each person has been experiencing symptoms, the second version of the question lets the respondent know that you’re interested in the specific time at which the symptoms first occurred.

Pay Attention to Length

We often want to collect as much information as possible when conducting a survey. However, extremely long surveys quickly become tedious to answer. Participants get tired or bored which in turn decreases the completion rate. After a while, you risk getting inaccurate responses from people who are trying to finish the survey as rapidly as possible. When reviewing your questionnaire, ask yourself, “Do I really need this question? Will it bring any valuable data that can contribute to my reason for doing this survey?”

Track Progress

It is good to indicate the respondents’ progress as they advance through the survey. It gives them an idea of their progression and encourages them to continue on. In a self-administered survey, a progress bar such as the one below could be placed at the beginning of each new section or transition.

In an administered survey, the interviewer can simply make statements such as:

“We are halfway done.”

“There are only two more sections left.”

“Only a few more questions to answer.”

Train Interviewers

If you are conducting an administered survey, you will have to prepare your interviewers ahead of time, explain their tasks, and make sure that they understand all of the questions properly. You may also want them to be supervised when they are first conducting the survey to monitor for quality to control for interviewer effects.

Pretest

Pretesting means testing your survey with a few initial respondents before officially going out in the field. This allows you to get feedback to improve issues you might have with length, technical problems, or question ordering and wording. Sometimes this step gets skipped if there’s a tight deadline to meet, but you shouldn’t underestimate its value. It can save a lot of time and money in the end.

Anonymize Responses

We briefly mentioned response anonymization earlier when we talked about questionnaire structure. Sometimes, people are afraid to give an honest response for fear that it might be traced back to them. By anonymizing responses you will get more accurate data while protecting the identity of your participants. There are times when you may need to keep respondents’ contact information associated with their responses, like if you’ll be conducting a follow-up survey and will need to know what their initial answers were. Be clear in your survey about whether you’re anonymizing responses or not so respondents know if you’re storing their contact information.

You should always anonymize any results you present, whether the data you collect are anonymous or not.

Incentives

The use of incentives can be contentious. You want to encourage as many people as possible to participate in your survey, but you also don’t want people completing the survey just to get a reward. Larger reward amounts can also raise ethical concerns of participant coercion (i.e., having individuals participate when they would otherwise refuse due to personal discomfort, risk of repercussions, etc.).

If the survey is long and time consuming, consider giving a small reward or stipend if a respondent completes the questionnaire. If the survey is shorter, consider doing a prize drawing from respondents who complete the survey and choose to provide their contact information.

Wrapping Things Up

As you have noticed, there are a number of factors you will need to consider when you’re designing your survey and deciding what types of question to use. Survey creation can seem daunting at first, but remember this is the foundation of your data analysis. Getting this step right will make the rest of your work much easier later on.

For further reading, please see our resources section.

</div>

Additional Data Collection Methods

第五章

更多的資料蒐集方法

By Dyanna Gregory

</header>

並不是所有的資訊都能用問卷調查輕易取得，問卷調查是在某一時間點自行回報的一個工具，沒有精確測量，也很難補捉外界對其反應的印象以及一段時間內的資料。然而，這些部份可能是你需要的重要資訊，讓我們來討論一下能更好的蒐集這些類型資料的其他工具。

方法
適用於：

直接測量
需要精確數值；
自行回報的資訊容易有偏差

焦點團體訪談
還不知道到底想問些什麼；
想知道各層面的反應或有多個主題；
想知道團體動態；
想知道決策過程

觀察
你所要測量的能夠容易且公開的觀察到；
你想記錄參與者的行為

檢視現有文獻
你有興趣的資料已經在別處有記錄(例如收據，網路報表等等)

日誌
需要長期追蹤的變數；
事後回憶容易有偏差的資訊

直接測量

如果你想取得精確無誤的統計，有些變數應該採用測量而不是調查。例如，問卷調查很難甚至幾乎不可能取得許多正確的醫學變數。好比你在研究開始時需要蒐集參與者體重的資料，有一些可能回報不正確數字的常見原因。

缺乏資訊：當被問到時他們並不真的知道自己的體重

社會期望：他們覺得有一個 "正確" 答案是他們應該給的

隨意回答：他們知道自己的體重但不覺得需要回答那麼精確； “我大概170磅，就說這個數字吧。”

是否需要精確資料取決於你要如何使用這些資訊。如果你不是很在乎精密的測量，估計值就行，那只要問的問題是一般人能夠合理估計的，問卷調查可能也可行。如果你有一個變數，自行回報的資料可能不正確，而這些資料的即時性和正確性很重要，那就應該使用直接測量而不是問卷調查。

直接測量(direct measurement)，就是使用合適的裝置來測量變數然後把數值記錄到資料集裏。這常用在健康相關的變數上，因為一般人不會 “剛好知道” 。這些測量值通常可在表格上抓到，再將資料從表格轉到資料集中。

重要的是，你的表格必須清楚說明你希望測量值如何被記錄。你應該標明優先採用的單位以及你希望抓到多精密的測量值。一個簡單的溝通方法就是畫好特定數目的格子，這樣測量的人就知道你希望記錄到幾位數。

下圖是一個我們可能用來抓成人體重的表格範例。這裏我們希望用磅來測量並且記錄到兩位小數。

如果有關係，可以加一個地方來記錄是用什麼裝置測量。

焦點團體訪談

有時候看著別人回應你的問題，瞭解他們的思考過程，及觀察他們和別人的互動是很有用的。你也可能有廣泛多樣的主題想談而不是挶限於調查，或是你還不確定到底要問些什麼問題。在這些情況下 焦點團體訪談(Focus groups) 很有幫助。

基本的焦點團體訪談就像是開讀書會。一小群人﹝通常6到12人﹞聚在一起討論特定的主題，討論則由受過訓練的主持人帶領。主持人問出各式各樣的問題來得到更深層的意見而不是問卷調查那樣的單一答案。焦點團體訪談通常比問卷調查有彈性因為問的問題並不全是事先決定的。主持人有更多空間去發掘答案因為他可以根據回答的答案去設計新的問題。此外，因為參與者是在一個群體裏，其中一人的回答可能會讓其它人想出之前沒想到的答案。

然而，團體的設定和主持人在場都會造成偏差的參與者反應。在覆查及分析焦點團體訪談資料的時候把這些考慮進去是很重要的。

觀察

有時候你需要蒐集的資料就是觀察結果。讓我們再回到Fictionals冰淇淋店。你最近為店裏買了新傢俱，在考慮幾種佈置方式。你想看看那一種佈置最能配合客戶動線，於是你將傢俱擺成其中一種方式幾天，然後記錄下你對客戶在店內移動情況的個人觀察。之後再將傢俱改成另一種佈置也記錄下你注意到的事情。這些資料能幫你找出其他你可能想問的問題，或是做決定前還需要那些資料。

你可以用這種觀察方式洞悉自然行為。如果你的研究對象不是人而不能接受問卷調查，這就會特別有用：科學家一直都是靠觀察蒐集資料的！

這種方法一個主要的缺點就是在場觀察者改變了被觀察事件。每個人看事件都有自己的觀點，而他們報導一個事件也會被自己的觀點影響。要降低這種偏差，你可以多找幾個觀察者讓蒐集的資料能表現不同觀點。

檢視現有文獻

在某些情況下，你需要的資料已經存在其他文件中，而資料蒐集只需把資訊彙集一處。

做為Fictionals冰淇淋店的經理，你想看看最近六個月的銷售情形來瞭解近期新增的菜單是否有獲利。這個資訊已經可以從收據或POS軟體資料中取得。你只需要設定成方便你使用及分析的形式。

其他常用來匯編資訊的現有文獻包括書籍、報紙、網路日誌、以及網頁。也有一些完整資料庫可提供使用，這部份會在尋找外部資料那一章詳細說明。

當你使用其他文獻為資料的主要來源，你應該先建立一個資料蒐集計畫，就像你設計調查問卷時一樣。這個計畫應該詳細說明你要找那些資料，你想抓到那個測量層級，你需要的時間段﹝例如，你只要最近6個月的資料？還是12個月？﹞，以及你需要多少資料﹝例如，你想看全部的收據還是取樣？﹞。

如果有任何資料來源不是你自己的，一定要正確引用出處。感謝別人的工作成果很重要，日後若有人質疑你的資訊，也對支持你的研究很重要。

日誌

如果你要蒐集的資料是一段時間內發生在這個人身上的事，日誌表格很有用。要是你讓他們記錄下很多容易記錯或忘記的細節會特別有幫助。

你之前可能看過類似這樣的東西：

日誌(Diary) 表格通常用來追蹤飲食、藥物、或運動，但也可以用在任何你想追蹤一段時間的事情。他們可以是紙本表格或電腦化的，但一定要遵守和設計調查問卷同樣的原則。欄位應該妥善標示並清楚說明該如何﹝以及多頻繁﹞完成。

使用多種蒐集方法

當你考慮整個計畫，有可能並不是所有想研究的問題都能只用其中一種前面討論過的資料蒐集方法回答。你也許會發現你的問卷調查需要補充一些直接測量，或你可能需要焦點團體訪談的成員填寫日誌表格。

就像你能受益於組合的問卷調查型態，當資料能合適的由不同方法取得，你也能受益於合併多種資料蒐集方法。考慮那些途徑最接近你需要的資料，然後選一個預算和時間限制內最實際的選擇。

</div>

Finding External Data

第六章

找尋外部資料

By Jane Foo

Running your own study to collect data is not the only or best way to start your data analysis. Using someone else’s dataset and sharing your data is on the rise and has helped advance much recent research. Using external data offers several benefits:

Time / Cost | Can decrease the work required to collect and prepare data for analysis
Access | May allow you to work with data that requires more resources to collect than you have, or data that you wouldn’t otherwise have access to at all
Community | Promotes new ideas and interesting collaborations by connecting you to people who are interested in the same topic

Where to Find External Data

All those benefits sound great! So where do you find external data? To help narrow your search, ask yourself the following questions:

Scope
What is the scope of the data you’re looking for? What are the:

geographic boundaries?

specific data attributes (such as age range)?

time periods?

Type
What type of data are you looking for? Do you need:

statistics?

research data?

raw data?

data that have been collected using a specific method?

Contribution
How will the data contribute to your existing data analysis?
Do you need several external datasets to complete your analysis?

Public Data

Once you have a better idea of what you’re looking for in an external dataset, you can start your search at one of the many public data sources available to you, thanks to the open content and access movement that has been gaining traction on the Internet. Many institutions, governments, and organizations have established policies that support the release of data to the public in order to provide more transparency and accountability and to encourage the development of new services and products. Here’s a breakdown of public data sources:

Source
Examples

Search Engines
Google

Data Repositories
re3data.org
DataBib
DataCite
Dryad
DataCatalogs.org
Open Access Directory
Gapminder
Google Public Data Explorer
IBM Many Eyes
Knoema

Government Datasets
World Bank
United Nations
Open Data Index
Open Data Barometer
U.S. Government Data
Kenya’s Open Data Initiative

Research Institutions
Academic Torrents
American Psychological Association
Other professional associations
Academic institutions

If you decide to use a search engine (like Google) to look for datasets, keep in mind that you’ll only find things that are indexed by the search engine. Sometimes a website (and the resource associated with it) will be visible only to registered users or be set to block the search engine, so these kinds of sites won’t turn up in your search result. Even still, the Internet is a big playground, so save yourself the headache of scrolling through lots of irrelevant search results by being clear and specific about what you’re looking for.

If you’re not sure what to do with a particular type of data, try browsing through the Information is Beautiful awards for inspiration. You can also attend events such as the annual Open Data Day to see what others have done with open data.

Open data repositories benefit both the contributors and the users by providing an online forum to share and brainstorm new ways to study and discuss data. In some cases, data crowdsourcing has led to new findings that otherwise would have developed at a much slower rate or would have not been possible in the first place. One of the more publicized crowdsourcing projects is Foldit from the University of Washington, a Web-based puzzle game that allows anyone to submit protein folding variations which are used by scientists to build new innovative solutions in bioinformatics and medicine. And recently, Cancer Research UK released a mobile game called Genes in Space that tasks users with identifying cancer cells in biopsy slides which in turn helps researchers cut down data analysis time.

Non-Public Data

Of course, not all data is public. There may come a time when you have access to a special collection of data because of your status within a particular network or through an existing relationship. Or maybe you come across a dataset that you can buy. In either case, you typically have to agree to and sign a license in order to get the data, so always make sure that you review the Terms of Use before you buy. If no terms are provided, insist on getting written permission to use the dataset.

Assessing External Data

Let’s say you’ve found a dataset that fits your criteria. But is the quality good enough?

Assessing data quality means looking at all the details provided about the data (including metadata, or “data about the data,” such as time and date of creation) and the context in which the data is presented. Good datasets will provide details about the dataset’s purpose, ownership, methods, scope, dates, and other notes. For online datasets, you can often find this information by navigating to the “About” or “More Information” web pages or by following a “Documentation” link.

Feel free to use general information evaluation techniques when reviewing data. For instance, one popular method used by academic libraries is the CRAAP Test, which is a set of questions that help you determine the quality of a text. The acronym stands for:

Currency | Is the information up-to-date? When was it collected / published / updated?
Relevancy | Is the information suitable for your intended use? Does it address your research question? Is there other (better) information?
Authority | Is the information creator reputable and has the necessary credentials? Can you trust the information?
Accuracy | Do you spot any errors? What is the source of the information? Can other data or research support this information?
Purpose | What was the intended purpose of the information collected? Are other potential uses identified?

Finally, when you review the dataset and its details, watch out for the following red flags:

Details of data collection method not stated

No contact information

Unclear ownership of data

No authoritative entities or credentials associated with data collector or data

Terms of use or license includes details that raises questions (e.g. data can’t be used for scientific study)

Inconsistent or missing metadata

Large chunks of missing data without explanation or reference points

Raw data looks “too perfect”

Published articles challenge or question the dataset

Using External Data

So now you have a dataset that meets your criteria and quality requirements, and you have permission to use it. What other things should you consider before you start your work?

Checklist |
Did you get all the necessary details about the data? | Don’t forget to obtain variable specifications, external data dictionaries, and referenced works.
Is the data part of a bigger dataset or body of research? | If yes, look for relevant specifications or notes from the bigger dataset.
Has the dataset been used before? | If it has and you’re using the data for an analysis, make sure your analysis is adding new insights to what you know has been done with the data previously.
How are you documenting your process and use of the data? | Make sure to keep records of licensing rights, communication with data owners, data storage and retention, if applicable.
Are you planning to share your results or findings in the future? | If yes, you’ll need to include your data dictionary and a list of your additional data sources.

Your answers to these questions can change the scope of your analysis or prompt you to look for additional data. They may even lead you to think of an entirely new research question.

The checklist encourages you to document (a lot). Careful documentation is important for two big reasons. First, in case you need to redo your analysis, your documentation will help you retrace what you did. Second, your documentation will provide evidence to other researchers that your analysis was conducted properly and allow them to build on your data findings.

Giving Credit to External Data Sources

Simply put, crediting the source of your external dataset is the right thing to do. It’s also mandatory. Ethical research guidelines state that crediting sources is required for any type of research. So always make sure that you properly credit any external data you use by providing citations.

Good citations give the reader enough information to find the data that you have accessed and used. Wondering what a good citation looks like? Try using an existing citation style manual from APA, MLA, Chicago, Turabian, or Harvard. Unlike citations for published items (like books), citations for a dataset vary a great deal from style to style.

As a general rule, all styles require the author and the title. In addition, editor, producer or distributor information (location, publication date), access date (when you first viewed the data), details about the dataset (unique identifier, edition, material type), and the URL may be needed. For government datasets, use the name of the department, committee or agency as the group / corporate author.

For example, let’s say you’re using the U.S. Census Annual Survey of Public Employment and Payroll.

The APA Style Manual (Publication Manual of the American Psychological Association, 6th edition) would cite this the following way:

while the MLA Style Manual (MLA Handbook for Writers of Research Paper, 7th edition) cites the same census data as:

Data repositories and organizations often have their own citation guidelines and provide ready citations that you can use “as is”. The Interuniversity Consortium for Political and Social Research (ICPSR), The National Center for Health Statistics, Dryad, PANGAEA, and Roper Center Data all provide guidelines for citing their datasets.

This chapter gives you a brief look into external data: the important takeaway is that we are only at the start of a significant growth in data thanks to the technologies that now make massive data storage and processing an affordable reality. Open datasets in particular have the potential to become a de facto standard for anyone looking for data to analyze.

</div>

Preparing Data

準備資料

當你收集了食譜的材料，在烹飪之前，你需要先處理一下這些材料。在準備的步驟中，你可能會洗、切、或任何方式，使你的食材準備好。因為這個步驟不太有趣，大多數的人並不喜歡這個步驟，但如果你曾經吃過一盤骯髒的青菜，你就知道如果你草率、或甚至整個跳過處理步驟，會受到多大的影響。

就像在烹飪一樣，許多人在處理資料的時候，也不喜歡準備步驟，但這是必須的。你不可能只撈到一些資料，丟進視覺化的城市，就預期它會產生正確的結果。如果你想要盡你所能的做出最好的圖像，你需要確認你的資料已經準備好。有些資料需要切分成不同的群組，有些可能需要轉換，還有一些需要清理，就像蔬菜一樣。

在這個段落，我們會說明一些處理資料的時候，常見的準備、清理工作，也會介紹一些可以減少花費處理時間的方法。不管你如何分割數據，如果你正確的處理你的數據，當你在處理可視化階段的時候會順利許多。

Getting Data Ready for Cleaning

Chapter 7

Getting Data Ready for Cleaning

By Oz du Soleil

One aspect of data that is difficult both to learn and to teach is how to get your data in shape so that it’s useful. There are a few common preparation tasks that you might encounter when working with data, especially if you’re handling data someone else collected. While we can use computers to perform many of the actions required for data preparation, there is a certain amount of manual involvement in figuring out exactly which tasks need to be done. For this reason, many people admittedly hate this part of working with data, but someone has to clean this stuff up!

Whether you’re a highly-paid data analyst with billions of records or a 1-person business with a 90-person contact list, you’ll be faced with messy data at some point. Unfortunately, data preparation isn’t a straightforward task and there’s no one right way to go about doing it. Each dataset is unique and some techniques may be used just once in a lifetime, as you’ll see in the examples below.

Separating Data

The first part of data preparation is separating data into the fields that will be most useful to you.

Have you ever been given a dataset where you couldn’t directly access the information you needed? Maybe complete addresses were in a single field, preventing you from getting statistics about specific cities and states. Perhaps a vendor sent an inventory spreadsheet with 6000 numbered parts but their numbering system includes the warehouse code followed by the part number and you need the part numbers by themselves.

You Want | They Provide
C77000S | GA3C77000S
W30000P | GA1W30000P
D21250G | DE1D21250G

Consider the challenges presented in the dataset below:

Mall | Address | City | State
Warm Willows Mall Peters Road Marrison, MI |  |  |
Jaspers Martinson & Timberlake Rds Reed, FL |  |  |
Lara Lafayette Shoppes 17 Industrial Drive Elm, CT |  |  |

You want the mall name, address, city, and state to be stored in separate fields. The dataset has hundreds of malls in it so splitting the entries apart by hand would take a lot of time. The mall names are bold, which makes it easy to visually distinguish where the mall name stops and the address starts, but not all the addresses begin with numbers, so no standard tool exists for separating the data into different fields. Code can be written that will recognize the bold and plain font weights and peel them apart, but since this is a rare situation, it is likely that you won’t have it right at hand. This is a case where you might write code (or have someone write it for you) that you’ll never use for this purpose again.

We can’t teach you everything about how to split data in this chapter since every situation is unique. However, there are some strategies that are useful in many cases. We’ll go through a few of these, as well as some common challenges and their sources, to help get you started.

Let’s Begin

A straightforward example that many people encounter is separating first names from last names. You may receive a dataset where full names are contained in a single field and you need the first and last names to be separate, or there may already be separate fields for first and last names, but some of the entries have the full name in one of the fields.

All in one field:

Name

Keith Pallard

Fumi Takano

Rhonda Johnson

Warren Andersen

Juan Tyler

Cicely Pope

Two fields, but not all info is in the correct place:

First Name
Last Name

Keith Pallard

Fumi Takano

Rhonda
Johnson

Warren Andersen

Juan
Tyler

Cicely
Pope

When datasets come to us this way, the challenge is seemingly easy to resolve. There are common ways of pulling apart the first names and last names for Keith, Fumi, and Warren. The easiest way is to look for the space, break the name apart at the space, and voila!

This is simple enough, but when we add in reality, things get complicated quickly. What if, for instance, your dataset has thousands of records? You’ll be spending a lot of time manually separating names, and there are far more potential name combinations than just first and last name.

Middle Initials | Martina C. Daniels
Professional Designations | Lloyd Carson DVM
2-part Last Names | Lora de Carlo
Prefixes | Rev Herman Phillips
Suffixes | Jimmy Walford III
Hyphenated Last Names | Tori Baker-Andersen
Last Name First | Kincaid Jr, Paul
2-part First Names | Ray Anne Lipscomb
Prefixes/Suffixes | Rev Rhonda-Lee St. Andrews-Fernandez, DD, MSW
Other fields incorrectly included | Murray Wilkins 993 E Plymouth Blvd
No/Missing First Name | O’Connor
No/Missing Last Name | Tanya
No name at all |
I have no earthly idea! | JJ
Not a person’s name at all | North City Garden Supply

Now what? Let’s Make Some Decisions

Let’s say we’re faced with separating the names so that you can sort by last name and that our list has 500 names (too large to consider reformatting by hand).

Before starting, we need to know certain things:

Why is it important to parse this specific field? Is anything being hurt by having names all in one field?

What do we want the result to look like?

Is it important to keep ‘Rev.’ and create a separate field to hold other titles like Dr., Mrs., Capt., etc.?

Should ‘Jr’ stay with the last name or should there be a separate field for it?

Should middle initials be kept? In their own field? With the first name? With the last name?

Do we want to keep professional designations?

Is it worth the cost of getting it done? In simple cases, you may be able to hire someone else to complete the work, but this might not be ideal for complicated ones. What if a professional said they could fix the list for $1000?

What do we do with incomplete or incorrect entries?

These questions should be answered before you start working with your data. If you just dive in, you can create a larger mess, and these decisions can actually make the process much easier. For example, if you decide that the professional designations don’t matter, there might be an easy way to get rid of them and simplify other parts of the project.

Say you’re working with a list of 20,000 names, 19,400 from the US and 600 from Turkey. The honorific titles in the US come before the name (e.g. Mr. John Smith), whereas Turkish honorific titles come after the name (e.g. Jon Smith Bey). You’re trying to figure out if you need separate fields for the Turkish data or another dataset altogether, so you ask the client what their preference is.

Their answer is simple. They aren’t conducting business in Turkey and they don’t care if you delete those records. GREAT! Just 19,400 records left.

Now, how do we break up these data?

There are so many techniques, we could write an entire book on this subject alone. You can break apart the data in Excel, or if you have programming skills, you can use Python, SQL, or any number of other languages. There’s too much for us to cover in this chapter, but for some good references, check out our Resources Appendix. For right now, we’ll cover some basic starting strategies so you’ll be better equipped to take on those references when you’re ready.

You should always make a copy of your dataset before doing any data preparation or cleaning in case a change you make is incorrect and you need to refer to the original.

Look for Low-Hanging Fruit

In many instances, most of the data that need to be separated are fairly simple. Out of 500 names, you may discover that 200 have just a first and last name. Set these aside. Leave them alone. Then look at the remaining 300 names.

Identify the Oddities

Look through your dataset for the no-names, complicated names, incomplete names, non-person names and any entries you don’t know what to do with. Set these aside. We’ll say these comprise 40 more names.

Look for Similarities

Of the 260 remaining names, maybe 60 are complicated by professional alphabet soup after their names. Whether you’re deleting these professional designations or putting the designations in their own field separate from the name, work with them all at once. Now the designations are separate (or deleted) and you have a field with just names. For those that have only a first and last name, add those to the 300 that were set aside at the beginning.

We put the 2-part last names and other remaining types into their own groups.

Manual Effort

The 40 oddities you identified at the start may come down to just re-typing manually, depending what is odd about them.

Sometimes, when we’re handling data in real life, records take care of themselves. For example, maybe a non-name record turns out to be a duplicate of a record that’s complete. In that case, we can just delete the incorrect record and move on.

Commonly Problematic Fields

Depending on how data are collected, there are other fields you may be interested in splitting up besides names. Some of the most common are:

Address information

Phone numbers, particularly if you need the area code to be separate

Emails, if you’re curious about domain information

Date, if you only want year or month

Units and Unit Conversion

Another important data preparation task is making sure that all the data in a single field are given in the same units. Ideally, you would have specified the unit type on the input form, but you may be working with someone else’s dataset or the data may have been collected directly from other sources where the information was compiled by machines that measure in particular units. For example, you might have details from medical records from multiple countries and the patients’ weights could be in pounds for some records but in kilograms for others. It is important to convert all the numbers to either pounds or kilograms so that they are all on the same scale, otherwise the records cannot be directly compared to each other and any visualization you do of the original data would look quite strange indeed!

You should do a quick scan of the variables in your dataset to identify any fields that could potentially report multiple units so that you can figure out if these conversions are necessary. If they are, you might need to reference an additional field to identify what the original units are that you should be converting from so you know what records need to be changed. If there isn’t a specific field that lists the units themselves, other fields like geographic location can be helpful. If you cannot find a field that lets you know what the units are but you suspect that a conversion is necessary, you should contact the original provider of the data to obtain that information. It may turn out that you actually don’t need to convert anything and your data are just strange, but it’s better to check and make sure than to ignore a potential error in your data.

Another type of conversion that is sometimes less obvious is data type conversion. It is important to make sure that all of the data in a single field is being stored as the same type or your visualization might not appear correctly, depending on how your software interprets the information. For example, “80” might look like a number to you, but the computer might actually be storing it as a string of text rather than in a numeric form. Some visualization software will go ahead and treat any text that looks like a number as a number, but other software will not. It is a good idea to check that each of your fields (or variables) is stored as a single, specific data type, such as text or numeric. This will ensure that anything that is captured in that field is being treated as the data type that you mean for it to be.

Controlling for Inconsistencies

One of the most work-intensive tasks in cleaning data is handling inconsistent information. For example, we see multiple languages popping up in a single dataset more often now that the internet allows us to easily collect data from users all around the world. This can sometimes create problems when you’re trying to group things up for visualization purposes, depending on how your data entry form is designed and what data from the set you’re using. Perhaps you do a survey of college students and one of the text fields asks for their major. One student might enter “Math” while another enters “Mathematics” and another types “Applied Mathematics.” You know that all these are considered the same major on your campus, but a computer or visualization software would not group these records together. You would need to create a single controlled vocabulary term (e.g. change them all to say “Math” or “Mathematics”) or create a separate coded field if you wanted to have them treated as part of the same category.

Although the computer can aid in changing values, problems with data inconsistency often have to be handled semi-manually. If you are looking through text fields, much like when you do data coding, find and replace is your best friend. As long as you know what the main variants are of the data that you want to control, you can quickly change them all to be the same value.

Missing Values

One of the most frustrating problems is when data fields are simply blank or are incomplete. If data have not been collected, you may be able to return to the source to fill in what is missing, but you may also no longer have access to the source. It may also be possible that you do not know who the source was, for example, in the case of an anonymous survey. If you are not able to obtain the data, it is important to handle the missing values correctly. You should set pre-defined values that you can enter in where the missing values would be so when someone looks at your dataset, they know that the data for that field are actually missing and you didn’t just forget to type them in.

These values should fall well outside the range of your reasonable data so that they clearly stand for missing data and nothing else. For datasets that do not deal with negative numbers, “-9” is often used as this default value, at least in the numeric fields. “999” is another popular choice for datasets that do not use large numbers, and a single period (.) is used by some statistical software programs that may be used with datasets with both negative and large numbers. For text fields, a single dash (-) is commonly used as the indicator for a missing value.

Keep in mind that a missing value is not inherently the same thing as an intentional non-response! You don’t have the particular information that the question was asking about in either case, but when someone actively chooses not to answer, that in itself is a piece of data you wouldn’t have if the question were unintentionally skipped. Those data aren’t missing: you know exactly where they are, the respondent just doesn’t want you to have them. As discussed in the Survey Design chapter, it is good to include a “Prefer not to answer” option for questions that may be of a personal nature, such as race or ethnicity, income, political affiliation, etc. That way, you can designate a code for this type of response so when you are going through your dataset later on, you know the difference between the respondents that purposely chose not to provide you a given piece of information and the data that are just missing altogether.

It is important to note that for purposes of basic descriptive visualization, missing values are described by including a non-responder category or by noting a change in the sample size. However, in inferential statistics, missing values may be dealt with in a variety of other ways, from exclusion to imputation to full analysis methods like use of EM algorithms.

Minimizing the Burden of Data Preparation

The best solutions are preventive. If you are the one creating the form for user input, do whatever you can to prevent receiving data that will require intensive handling during the data preparation stages. In the Types of Data Checks chapter, we’ll talk about different strategies for minimizing the number of data preparation tasks that need to be performed.

If you’re not the one collecting the data but can speak with the people who are, try to work with them to identify and resolve any problematic data collection points using the strategies in the Types of Data Checking chapter as a guide.

Post-Data Preparation

Once your data are separated into the fields you want, converted into the units and types you intend to work with, and terminology is controlled, you’re ready to proceed to the data cleaning phase where you’ll check for actual errors in the data. In the next three chapters, we’ll talk about the basic process of data cleaning, different processes for performing checks on your data to find problems, and what you can and can’t do with data cleaning.

</div>

Data Cleaning

Chapter 8

資料清理Data Cleaning

By Marit Brademann and Dyanna Gregory

現在你手上有一個資料集(dataset)，你準備好清理它了。但這是什麼意思呢？資料清理(data cleaning)具體而言是什麼，我們又要如何進行呢？

清理資料時，我們要檢查並辦別出不正確的資訊，像是錯誤的數字或是錯字等，然後決定是否要修正它們(如果它們可以被修正)，或是直接刪除它們。如同資料準備的過程，許多資料清理 的任務同時包含電腦計算和人工檢查兩個步驟。人工檢查尤其重要，因為我們需要進一步確認電腦所判斷為可能錯誤的資料是否真的為誤。有些電腦判斷為有問題的資料，可能只是較極端的數據，所以你應該持續參與資料清理的過程，不要讓電腦獨自運作，否則有用的資料可能會被刪除！

如何清理資料

讓我們從一些最基本的資料清理開始。接下來將用excel為例，但你也可以使用其它的制表軟體或是資料處理軟體來進行以下步驟。常用的程式列在附件中.

範圍檢查Range Checks

範圍檢查(Range checks) 是非常簡單明瞭的步驟，用來檢查數值資料集中的資料是否不超出邊界值，並且在可接受的數值範圍內。以下面的作業成績為例。假想你是一位大學教授，而你的助教輸入了學生們這學期的作業成績。你想要確保成績輸入無誤，因此你打開資料，並把第一個作業的成績(依1-100評分)該欄依大小排列。前幾排顯示如下：

學生ID | 作業一成績
679372531 | 980
673540288 | 99
674082892 | 97
673923590 | 96

這裡有一個成績顯示為980分，很可能是助教不小心在98分後面誤加了0。你應該標記它並且向你的助教確認正確的成績。

即使所有數值都在正常的範圍內，用檢視的方式進行資料範圍檢查，也可能透露其它可能的問題：

學生ID | 作業一成績
674472019 | 78
679029425 | 75
671822390 | 74
671278927 | 9

在這組成績裡，有一個成績特別地低，很有可能原本的成績是90。因此，它應該要被標記起來並檢查資料來源。這是一個很好的例子，說明了人工參與資料清理的過程是非常重要的，尤其是那些權重極高的變項，比如學生的成績或是一個研究計畫的主要變項。但是，人工檢查資料集中的所有變項可能曠時費日，尤其當資料集非常巨大時，而且，並不是所有變項都同樣重要。因此，你必須決定哪個變項需要人工特別檢查，哪些又可以交由電腦處理。

範圍檢查可以在任何範圍內執行， 即使小如0和1之間的小數點。比如華氏體溫，正常人的體溫多數位於97.0與99.0之間，有人偏高，有人偏低。透過範圍檢查 你可以檢查出變項裡的數值是否過低或過高。

你也可以使用「篩選(Filter)」指令來辨認一變項中所有超出正常可接受的數值。但是，該指令無法檢查出相對異常但仍在範圍內的數值。以上述作業成績為例，篩選(filter) 指令能辨認出980為異常，但無法判斷9為異常。如果你使用篩選進行範圍檢查，你最好同時使用其它方法來對變項進行總體檢查，如此才能過濾出所有可能異常的數值。

錯字檢查Spell Check

錯字檢查是另一個針對資料集是否有誤的基本檢查。我們建議你一項一項地檢查，而非一次性地檢查整個資料集。因為在一變項中被視為錯誤的寫法，在另一變項中極可能被視為正確。姓氏是個很好的例子，如果你的資料集有一欄為姓氏，電腦很可能警告許多姓氏是錯誤拼法，即便它們正確無誤。如果你一項一項地檢查，你將可以較快地檢查完整個資料集。在資料準備一章學生調查的案例中，學生們填寫了主修科目。假設有一個學生因為熬夜而誤將「數學」填成「樓學」，電腦只針對主修科目這個欄位進行錯字檢查時，將可很快地辨認出錯字，讓你能夠更正為「數學」或是「數學系」，看你選用哪一個字彙。

樣式比對/正規表示法Pattern Matching/Regular Expressions

另一個較為進階的資料檢查是樣式比對(pattern matching)。這種檢查法可以用來檢查諸如email地址的欄位。這種方法涉及正規表示法(regular expressions，又作regex)，這是一種表達方式，能讓電腦知道：「在這欄中我只要符合某種特定樣式的資料，所有不符合的都標示出來。」定義樣式的方法依不同程式而有所差異，如果你從未寫過這樣的程式，你可能會覺得它看起來有些複雜。但是如果你曾經在程式中寫過星號「＊」作為檢索的萬用字元，那正是正規表示法，你已經稍微認識它了。

在Excel中也有樣式比對的選項，和一些十分有用的進階篩選功能。參考資源中關於正規表示法、Excel篩選，以及樣式比對的更多細節。

組合檢查Combination of Fields

你也可以綜合幾個欄位進行檢查。有時，這樣的檢查是必須的，因為你需要同時檢查數個欄位才能判斷資料是否有誤。如果你有在使用線上銀行，你已經在不知不覺中進行組合檢查了。你的線上銀行顯示了許多資料，你必須同時檢視才能讀懂它們，如果有讓你感到疑惑的資料，你心裡馬上會有警覺。你的線上銀行會同時顯示交易對象、交易金額、使用貨幣、信用卡或debit card刷帳、交易日期和帳戶餘額。這些所有的資料都是這組資料集的一部分。當你登入你的線上銀行並確認所有資料正確無誤時，你正是在進行組合檢查。如果交易金額與你預期的有所不同，或是與餘額不符，你心裡將會有所警覺，並去電銀行查清問題點。

網路銀行和其它的資料集是一樣的，某些資料需要綜合起來檢視。假想你在針對醫院病患進行醫學研究，你追蹤病患每日服用的藥品並記錄三種資料：藥品種類、藥品劑量和劑量單位。舉例來說，資料「阿斯匹靈、500、毫克」代表該病患每日服用阿斯匹靈500毫克。想像你看到一筆資料是「嗎啡、200、磅」，你會有什麼反應？給病患服用嗎啡並不奇怪，而劑量200毫克也還是合理的用藥，因此數字本身並不會讓人疑惑，但是即使是一磅的嗎啡都有可能致死，顯然問題出在單位上。你應該會檢查病患資料，或是尋問輸入資料的人以確認正確的單位。

如果你的資料都是自由輸入，你將會得到無限多個組合。這時，你應該盡早檢查你的資料集，並判斷哪些資料可以一起進行組合檢查，那麼你便可以定期地檢查資料是否有誤。此外，因為組合檢查是一個費時的工作，你必須先判斷這個資料組合的準確度有多重要，看是要產生一個準確的視覺圖表，或是進行統計分析，或只是一個初略報告。

沒有進行資料清理的後果是？

因為許多資料清理的過程非常費時，你經常需要先決定哪些變項值得清理，以及應該使用哪種清理方法。但如果我們跳過資料清理的步驟，讓它仍是「髒資料」，會有什麼後果呢？這個問題很難回答，因為這取決於你的資料集有多「髒」。運氣好的話，你的資料集並不太髒，而你的結果報告所受影響不大。運氣不好的話 你將會因為資料集裡的錯誤沒有在資料清理中被修正，而得到錯誤的結果。

當然，你的資料可能一開始就處於很乾淨的狀態。如果是這樣，你也可以跳過資料清理的過程，並得到影響不大的結果。無論如何，在你對資料進行基本而全面的檢查之前，你並無法得知你的資料是否乾淨或是骯髒。這就是說……

承認吧！多數的資料集都不乾淨

清理資料就像是打掃房子，你永遠無法清掃到每個部分。不論你如何努力，你都無法強迫其它人輸入百分之百正確的資料，即使是我們自己也經常在處理資料時犯錯。雖然你希望你的資料能夠愈正確愈好，但角落裡總還是會有細小的灰塵。重要的是，你應該承認沒有一個資料集是完美無瑕的，並且培養出判斷力 ，讓你能判斷什麼樣的乾淨程度足夠服務你的目的。

舉例來說，你從擁有一萬筆聯絡資料的資料庫中取出一千個名單，並且發現這一千個名單中有兩筆資料把姓與名放在同一欄。你會試圖回頭檢查並修正這一萬筆的資料嗎？

不一定。

你應該針對你的目的來判斷資料足夠乾淨或是過於骯髒。這完全取決於分析的變項，以及你的目標結果為何。如果你負責為一個內部報告來檢查學生填寫的主修科目，而且你認為錯誤率只有0.01%，那麼這些錯誤可能並沒有那麼重要。但是，如果你的變項是一個安全報告的關鍵因素，且擁有5%的錯誤，那麼它的重要性便比上述主修科目還高。慢慢的，你將會對資料的乾淨與否培養出較好的判斷力，並且知道不同的變項組合需要花多少勞力在資料清理上。到了這時，你最好諮詢該案成果的主要關係人，以知道對他們來說各個資料變項要多乾淨才是合理的。你應該盡可能追求資料的乾淨，但永遠記得它無法完美無瑕。

資料準備和清理是有代價的。如果你花錢請人做，那麼代價是金錢。如果你自己做，那麼你將花費自己或是組員的時間(也許還有精神)。所以，如果你從未用過電話或是傳真這類的機器，或是你寧願仰賴上帝，你也許可以決定直接刪除資料，停止收集它們，或是乾脆不去煩惱資料清理。

資料清理之後：請分類並歸檔！

當我們清理好資料後，我們將會面臨全新的問題：我們(以及其它人)如何確認我們對資料的修正是正確無誤，並且沒有破壞資料呢？畢竟，經過處理的資料可能會與原始資料大為不同。

最簡單的解決方法便是記錄每個步驟，尤其是當你稍後可能與其它統計學家或是研究者分享你的資料。在清理你的資料時，你最好把每一個改變存成分開的檔案，這樣一來，你總是可以回頭比對修改後的資料和原始檔案，看看哪些欄或列已經被刪除等等。這樣也確保當你想要用不同的方式切割和清理資料時，你永遠可以回溯到處理前的檔案。

你應該在清理資料的過程中，小心撰寫參考指南，記錄下每一個辨別壞資料的步驟，以及哪些資料被刪除了。這個關鍵在於一邊清理資料，一邊撰寫紀錄，因為同步進行記錄比事後再透過記憶追溯要容易得多。如果你使用的軟體是像excel一樣，各個指令分開輸入、執行，你應該特別注意記錄的工作，因為你的每個步驟都是手動執行，不像程式語言可以輕鬆地重覆回放。判斷一個紀錄好壞的祕訣是，如果你無法容易地照著你寫的紀錄，重新做出相同的結果，那麼你不該期待其它人也做的到。

不管你決定如何做資料清理的紀錄，寫好它可以合理地說明為何你刪去某些資料，而和保留了其它資料。同樣的，其它人也可以檢查你清理資料的過程是否恰當且正確。除此之外，如果你認為你稍後可能會將你的資料分享給其它統計學者來做進一步的分析，提供原始資料和資料處理紀錄可以讓大家的工作更輕鬆、快速。

</div>

Types of Data Checks

第九章

資料檢查的類型

By Ian Tebbutt

在上一章，我們討論了資料清理和它所需要的步驟。在這一章，我們將會更深入討論資料檢查，與其它進行資料清理前後的驗證過程。

要對資料傳達的訊息產生信心，資料檢查是非常重要的過程。基本的資料清理步驟非常直接：假設你手上有很多筆資料，那麼每個欄位都應該有相對應合理的數值。舉例來說，年齡應該在0到120歲之間(而且多數人在80歲以內)；交易日期應該要在近期，通常在過去一兩年內，特別是當你處理的是網路(比方Twitter)所產生的資料。

無論如何，資料檢查雖然容易理解且十分重要，卻是個複雜的工作，因為資料產生錯誤的方式非常多種，資料也可能以預期外的格式呈現。

資料檢查的時機When to Check

舉例來說，有一個關於某電信公司顧客變更電話號碼的資料集。該公司從上百家小型的手機服務業者匯集這些資料，卻沒有對它做進一步的檢查，但是這個資料集仍然每日被使用著。這是個資料檢查的典型案例，假設你要追蹤不同年齡使用者的電話費，下面的例子呈現了這個資料的一些問題：

電話類別一欄混合了代碼和文字

年齡欄中，0代表年齡未知，但會造成計算平均年齡的誤差；此外，112是正確的年齡嗎？

記錄編號 | 電話類別 | 年齡 | 新客戶 | 價格
1 | MOBILE | 0 | NO | $12.45
2 | Mobile | 47 | Y | 12 45
3 | Land Line | 34 | YES | 37
4 | LandLine | 23 | YES | 1.00
5 | LL | 112 | Y | $1K

資料檢查最基本的原則是即早檢查、經常檢查。即早檢查指當資料最初被輸入時，就立即修正它們。例如，如果新客戶一欄應輸入Yes或No，但卻被鍵入其它資料，像是A或是空白鍵，那麼應即時提醒使用戶修正資料。如果沒有及早驗證，錯誤的資料可能會進到資料庫，即使你發現錯誤，在沒有向使用者確認正確資訊前也難以修正。有時候，你可以透過比對其它資料庫裡的相同欄位尋找對的資料來修正原始資料。但這是個 複雜的過程，也容易導致其它問題，因為你必須判斷另一個資料庫裡的資料是正確的。

如果你能夠順利地控制資料收集的過程，那將會帶給你極大的好處，因為資料檢查最容易的方法便是在資料輸入時立即檢查。這種類型的資料檢查又稱作「前端檢查」或是「客戶端檢查」，因為它發生在使用者正在輸入資料，而資料還未進入到資料庫時。最常見的方式，是將收集資料的程式或網頁設計成只接受有效的資料格式。你以往在網路上填寫表格時，應該已經遇過這類型的資料檢查。

例如，國家和州應該從建立的列表中選取。如果你處理的是國際性的資料，那麼州的選項應該跟著所選國家而改變。

這樣一來，你的系統只允許對的資料格式被輸入。不過，這個方法並非完美。網路速度受阻的典型狀況，即是因為填入的資料直到最後提交時，才檢查出最初的輸入有誤。比較好的方法是在每筆輸入時都做檢查，但這個方法的缺點是程式比較難寫，並且會導致檢查指令連續不斷地送到伺服器端，同時資料有誤的警訊則不斷回送到使用者端。折衝的方法是，在流覽器一端先進行簡單的資料檢查和驗證，把比較複雜的部分留給伺服器端檢查。某些資料的檢查只能在伺服器端進行，最常見的像是安全性檢查，例如信用卡驗證。

為了縮短資料準備的時間，其它好的調查表格設計有：

先決定好你希望姓名如何被輸入。例如，使用者是否可以在姓名後加上小(Jr)、獸醫師(DVM)、博士(PhD)、會計師(CPA)等稱謂，或你希望稱謂與姓名分開輸入。如果是後者，你是否希望職業別的稱位與姓名後綴，像是小、二世、三世分開輸入？還有，你是否希望姓與名輸入在不同欄位？

將電話與日期的欄位設定成只能以你指定的方式儲存(本章稍後會對日期的部分詳述)。決定你是否要收集辦公室電話的分機號碼，如果要，幫分機另外設一個欄位。

不要相信任何人

不管你的表格設計得多完美，也不論你在前端用了多少驗證的程序，經驗告訴我們絕對不要相信使用者輸入的資料。如果資料是由人所輸入的，那麼就有可能發生錯誤。即使已經有做使用者端檢查，也一定要在資料送出後進行伺服器端或後端檢查back-end checks。這樣做有很多好的理由。比方說，如果你並沒有設計資料收集的工具，而你的資料來自不同的前端程式，有些有進行完善的用戶端檢查，有些則無，因此一些不乾淨或沒檢查的資料，就在整合這些資料集時進入你的系統。本章之初所舉的電信公司的例子，就是因為擁有太多的資料提供者，彼此因疏忽而產生紊亂的資料集。只要在前端多付一些心力，將能節省我們的時間，減輕我們清理資料時的挫折，並且提供我們較乾淨的資料集。

第二個黃金定律是，非到必要時，不設計文字輸入的欄位。舉例來說，某些國家習慣將地址資料分成數個欄位，例如第一行、第二行、城市、州、國家、郵遞區號。但在英國收集地址時，通常只需輸入郵遞區號以及門牌號碼，透過這兩個資料便可以查到正確的地址。用這個方法，不但郵遞區號可以被自動驗證，而且地址資料因為不是使用者以文字輸入，也能確保它的正確性。但在其它國家，我們仍需使用文字欄位，連帶地也產生許多檢查的手續。

資料中的逗號會產生許多問題，因為許多資料格式是用逗號來區別不同的欄位(即CSV逗號分隔值)，一個額外的逗號會產生多出來的欄位，導致原有欄位的資料被向右移。為此，最好不要直接從程式中剪貼資料，而是另存檔案，然後用另一個程式來讀取這個檔案。

稱號 | 名字 | 姓氏 | 地址第一行 | 地址第二行 | 城市 | 州 | 國家 |
Bill | Short |  | 13 | A The Street | Hastings | VIC | AUS |
| Mr | William | Tall | 27 The Close |  | Guildford | VIC | AUS

上列第二筆資料，因為多了一個逗號，使得資料往右邊移了一欄。這個錯誤雖然容易用人眼辨視出來，但是對於自動化的系統卻十分困難。一個好的檢查方法是人工查看是否有資料超出了最後一個欄位。這個例子也強調結合電腦和人工檢查資料的重要性。有時，快速地用眼睛掃描過資料，能夠讓結果大為不同。

資料格式和檢查

處理數字資料時，你應該注意幾個問題。這些數字是否合理？如果你處理的是金錢，一筆$1,000,000是正確的價格嗎？或是有人輸入了錯誤的數字？同樣的，如果價錢是0或是負數，是否代表這個產品是被贈送出去的，或是有人付錢來移除它？此外，為了在會計上平衡收支，許多價格資料會用負數表示。

除此之外，檢查出數字資料中的空白和字母也很有幫助。不過，貨幣以及負數可能會讓操作變得困難。如下例所示，這些是各種表示貨幣的有效方式，裡面包含了非數字的符號。

$-1,123.45
(1123.45)
-US$1123.45
-112345E-02

數字中帶有字母並不一定有誤，同時，負數也有不同格式。

日期的呈現也有許多值得檢查的問題。第一個問題是國際間存在著不同的日期格式。如果日期顯示為1/12/2013，在美國代表2013年1月12日，但在英國則是12月1日。幸運的話，你可能會取得國際通用的格式，顯示為2014-01-12，這種標準化格式的額外好處是，即使它儲存為文字，依然能夠被排序 (http://whatis.techtarget.com/definition/ISO-date-format)。無論如何，你可能沒那麼幸運，因此檢查並確保你知道日期是用什麼格式是非常重要的，尤其是當你的資料輸入者來自於不同國家且使用不同格式時。如果你在設計資料輸入的表單，一個解決日期格式的好方法，是設置月曆讓使用者從中選取日期，而非手動輸入。或者，你也可以在表格旁，指明你要的日期格式給使用者參考。

另一個可能遇到資料檢查的情況，是在分析並驗證他人的資料，以確認該視覺圖表和數據的正確性時。這有可能發生在工作上，當你需要證明其它人的分析結果，或是在線上看到公開的圖表，且附有原始資料讓你可以試著進行自己的分析。在這兩種狀況下，第一步可以先快速檢查每個數據的總和。接著，再以批判的眼光檢查視覺圖表，比如，這個圖表是否能支持原本的論證，或是與之相抵觸。除了需要將錯誤檢查出來，還必須判斷視覺圖表是否能有效地幫助理解。檢查他人的資料不但能夠增加你資料檢查的經驗，同時也是查看他人分析報告時必須先進行的工作。

資料版本Data Versions

另一個資料檢查過程中會遭遇的問題來自資料版本。

當程式和系統隨時間變化，資料欄位會增加、移除、甚至連資料收集的目的都會產生改變。舉例來說，澳洲的郵遞區號是四碼，以四個字元來儲存，但是其它系統已經改用更準確的五位元SLA地區代碼。當這些來自不同系統的資料整合在一起時，五位元的SLA經常被刪減以符合四位元的郵遞區號。要檢查出這種資料變化並不容易，雖然郵遞區號和SLA地區代碼都有公開對照表，但是，要判斷為什麼有些四位元的地區資料無法符合兩種代碼，需要費一番精力。

某些資料欄位與你的視覺圖表或最終報告雖無直接關係，卻能提供你關於資料收集的重要訊息，你也應該考慮收集這些資料，比方說這些資料是何時被建立的。如果在資料集被建立後，才增加新的欄位，原有的資料在這個新欄位應是空白的。這時，如果有人沒有妥善地處理資料，並把空白欄位以0取代，這將會拉低該欄的平均值，並嚴重影響你的圖表。此時如果你有資料建立的日期，你將可以查出誤輸為0的欄位，改正為「缺失資料」(missing data)，以此來修正資料集。同樣的問題也可能發生在當部分資料被刪除時。刪除的情形相對少見，不過有時資料會因時間推移而改變其收集目的，如果目的改變了，理解該資料將會極具挑戰性。

金額 | 付款方式 | 伺服器ID | 建立日期
$100 | CC |  |
$143 | Cash |  |
$27 | Amex | 237 | 3/1/2013
$45 | Cash | 467 | 3/1/2013

以上表格呈現了原有資料加入兩個新的欄位—伺服器ID和建立日期—後的樣子。新欄位很有可能是在2013年3月1日建立的，因此，如果你的分析僅限於該日以後的資料，那麼你可以從這個資料集追蹤銷售點或伺服器。但是，這個資料集沒有該日以前的伺服器ID，所以如果你想要看2013年1月1日的資料，你就必須從其它地方找尋資料。

資料檢查另一個重要的問題，是資料欄位與其數值的意義可能因為時間而改變。在最完美的狀況下，資料所有的改變都被詳實地記錄下來，因此你會正確地知道每一筆資料的意義。但現實的情形是資料的改變很少被完整記錄。因此，另一個理解資料真正意義的好方法，是向處理資料的行政人員以及操作系統的使用者請教。

以上僅是一些讓你能夠清楚了解你的資料，並且意識到可能的錯誤的步驟。在下一章，我們將討論其它混淆資料的隱藏性錯誤，和判斷這些潛在問題的方法。

</div>

What Data Cleaning Can and Can’t Catch

Chapter 10

What Data Cleaning Can and Can’t Catch

By Dyanna Gregory

Now that we understand what data cleaning is for and what methods and approaches there are to shape up our dataset, there is still the question of what cleaning can and can’t catch.

A general rule for cleaning a dataset where each column is a variable and the rows represent the records is:

if the number of incorrect or missing values in a row is greater than the number of correct values, it is recommended to exclude that row.

if the number of incorrect or missing values in a column is greater than the number of correct values in that column, it is recommended to exclude that column.

It should be made clear that exclusion is not the same as deletion! If you decide that you don’t want to include a row or column in your analysis or visualization, you should set them aside in a separate dataset rather than deleting them altogether. Once data are deleted, you can’t retrieve them any longer, even if you realize later on that there was a way to fill in the missing values. Unless you are absolutely certain that you will not use a record or variable again, do not just delete it.

In the last few chapters, we have talked about several different processes for data cleaning and have seen the types of problems they can help identify and fix. When we’re searching for errors and mistakes, we are able to detect potential problems such as:

inconsistent labels, misspellings, and errors in punctuation;

outliers, invalid values, and extreme values;

data that aren’t internally consistent within the dataset (e.g. 200 lbs. of morphine);

lack or excess of data;

odd patterns in distributions; and

missing values.

What we haven’t talked a lot about yet is what data cleaning can’t catch. There may be incorrect values that are nevertheless both within the acceptable range for the data and that make complete sense. For example, if someone enters the number 45 instead of 54 into your dataset and your valid range of numbers is 0-100, it will be unlikely that you’ll catch that error unless that field is one that you’re cross-checking with another field or you’re verifying the information with an outside source record.

Similar to that, you may be receiving information from an online survey form and the person filling it out may have selected the button for “Strongly Agree” when they actually meant to select “Strongly Disagree.” Again, unless this answer is somehow cross-checked with another variable or source, you will have no easy way to detect this error. Sometimes this type of error is more critical than others. If a person selects “Strongly Agree” instead of “Agree” on an opinion survey, that is unlikely to have the same impact on the results as if someone accidentally marks the wrong gender on a form for a research study where you are using gender as a grouping category for treatment assignments.

Data cleaning also can’t tell if a missing value is truly missing (i.e. the question was accidentally skipped or the data were not collected for some reason) or the question was purposely skipped (i.e. the participant declined to answer) unless “Prefer not to answer” was an answer choice. This may be relevant in some cases (particularly in demographics), though in others, you may decide to just treat both as missing data. This is why, as mentioned before, you need to include a “Prefer not to answer” choice for any question of a personal nature where you want to know if the data are truly missing, since some people may actively choose to not answer questions about race/ethnicity, income, political affiliation, sexual orientation, etc.

</div>

Data Transformations

Chapter 11

Data Transformations 資料轉換

By Kiran PV

This chapter covers more advanced statistical concepts than some of the others but we wanted to include a brief introduction to data transformations in case you encounter them. If you need to do your own transformation, check out the resources in our Appendix for additional tips.

相較於其他章節而言，本章節涵蓋較多進階統計概念，但我們想將資料轉換的簡介納入待用。若您需實做資料轉換，可參閱 附錄，有更多的資源及指點。

When you take a digital photo, sometimes the picture comes out in a way that makes certain features hard to see. Maybe the colors are too dark or too light, the photo is blurry, or the objects in the image are too small and you want to zoom in on them. You can pop the photo into something like Instagram or Photoshop, tweak it, apply a filter or two, and transform the picture so it’s much clearer and easier to understand.

有時，以數位相機拍下相片之後，才會發現相片中的某些重點難以辨識。也許是因為色調過暗或過亮、成像模糊、主題物品太小必需放大，諸如此類。為了讓相片主旨更加達意，一般常見的方式是把相片載到 Instagram 或 Photoshop 之中加以調整、加上一兩層濾鏡，或是裁切修改等等。

Sometimes we need to transform data, too. You might get a set of data where, if you visualize it as is, there will be parts that are difficult to see. Also, if you’re going to do statistical testing of your data, many common tests make specific assumptions about the distribution of the data (e.g. that the data are normally distributed). In the real world, we often come across data that don’t meet these assumptions. Data transformations are one way to handle both of these problems. Here, we’ll talk about some of the more common transformations so that when you encounter these terms in the future you’ll have an idea what was done to the data.

資料的轉換也其實偶有需求。你可能從某處取得某些資料集，在視覺化處理之後，依然會有有難以呈現的部份。若要對資料集進行統計測試，更有許多測試方法只能適用於其假定的分佈（像是常態分佈）。在真實世界中，我們取得的資料集，經常無法對上這些假定條件。資料轉換，則可以用來對付前述兩類問題。在本篇章中，為了讓你在未來與到這些問題時稍有概念，我們會盡量談及常被使用的轉換方法。

Data transformations are one of the common manipulation procedures which can reveal features hidden in the data that are not observable in their original form. We can transform the distribution of data to make it easier to see and so that any required assumptions of statistical tests are met. We usually do this by replacing one variable with a mathematical function operating on that variable. For example, you could replace a variable x by the logarithm of x or by square root of x.

在資料集中，無法直接觀察得知的重點，透過 資料轉換 這種手段， 操弄之後，便能將資料集中所隱藏的重點彰顯出來。我們亦可以調整資料分佈，使其較容易被觀察，或使其能滿足統計測試所需的前提。一般的作法，是將其中某項變數代換為該變數的一元函數。比方說，將變數 x 換為其對數，或是取其平方。

Never perform the transform directly on your original data! Either create an additional column to hold the new values for each variable you’re transforming or make a copy of your entire dataset.

注意！進行轉換時，切勿直接修改原始資料。務必將原資料集另行備份，或是增加新欄位來存放變數轉換之後的結果，不要直接更改原始變數。

常態分佈與資料的傾斜度 Normal Distribution and Skewness in Data

One of the most frequently-encountered assumptions of statistical tests is that data should be normally distributed. You may have heard of the normal distribution referred to as a “bell curve” before; this is because a normal distribution takes the shape of a bell, with the data spread around a central value. Some of the data examples that commonly follow a normal distribution are related to human measurements such as height, weight, life span, and scores on IQ tests.

在統計測試中最為常見的前提，便是資料應為 常態分佈。您可能看過「鐘形曲線」這名詞被用來代稱常態分佈，那是因為常態分佈確實畫出來像個鐘，資料的值集中在中間。人的身高、體重、壽命、智力測驗分數等等，這類測量資料，一般都認為是常態分佈。

Unlike a normal distribution, which is symmetric around the mean value, skewed data tend to have more observations either to left side or to right side. Right skewed data have a long tail that extends to right whereas left skewed data will have a long tail extending to the left of the mean value. When data are very skewed, it can be hard to see the extreme values in a visualization. If you notice that your data distribution is skewed, you might consider transforming it if you’re doing statistical testing or if the data are difficult to visualize in their original state.

常態分佈，是以其平均值為中心，呈現對稱的形狀，相對而言，偏斜的資料 則會在一側得到較多的觀測資料。向右偏斜的資料集，會有長尾向右方有延伸，而向左斜的資料集的長尾則是向左方延伸。當資料非常偏斜之時，就算透過視覺化呈現，也很難看出其極端的值為何。若你需要進統計測試、或視覺化的工作，在注意到手邊的資料分佈有點偏斜之後，可考慮進行轉換。

常態分佈 Normal Distribution

左偏斜 Left Skew

右偏斜 Right Skew

以抽樣來了解轉換效果 Understanding Transformations Using Sample Data

Let’s use the population and land area of the 50 US states from 2012 to see how transformations work on actual data. The first step in transformation is to evaluate the distribution of the data. Then you can decide what transformation is appropriate (if one is needed). We can start by constructing a histogram of the population data and a scatterplot of the population-area data to get a better sense of how they’re distributed.

接下來我們使用美國五十州的人口與土地的真實資料做為範例，來進行資料轉換吧。首先，是評估一下資料的分佈，如此方能推知轉換是否合適（或必要）。為了更進一步了解資料的分佈，我們可以先建立人口資料的直方圖，與人口面積資料的散布圖。

The histogram above shows that the distribution of population values is right skewed. This is reasonable to expect because the majority of states’ populations lie in the range of 1-10 million. If we want to do statistical testing that relies on a normal distribution assumption, these data will need to be transformed.

上面這張直方圖，顯示人口數目的分佈向右偏斜。這算是合理，因為多數州的人口總數都落在一到一千萬之間。若我們想對此資料進行的統計測試，有要求資料集應為常態分佈的話，那這項資料進就必須先經過轉換。

In the scatter plot above, you can see that most of the data points are clustered in the bottom left corner of the graph, making it hard to see how population and land area are related. We can’t just scale the graph differently to “zoom in” on that corner because we’d knock California and Alaska off the chart. We can, however, use transformations to help make the data easier to view.

在這張散布圖當中，明顯可見到多數的資料點都聚集在圖的左下角，人口數與土地之間的關聯性，也因此較難以觀察得知。雖然可以對左下角做局部縮放，但那麼做的話，加州及阿拉斯加，就會超出圖表的範圍了。不過，為了讓資料觀察較為容易，我們可以進行資料轉換。

There are many transformation methods that can be applied in either of these situations, but let’s look at a couple of the common ones to see how they can affect both a visualization and the shape of a distribution.

有很多資料轉換的方法能對付這類情況，但現在我們就針對幾個常見的方法，看看它們對視覺呈現及資料分佈的形狀有何影響。

對數轉換 Log Transform

To do a logarithmic transformation, you calculate the log of each value in the dataset and use those transformed values rather than your raw data. Log transforms tend to have a major effect on distribution shape, and in visualizations can bring extreme outliers closer to the rest of the data so graphs aren’t stretched out as much. You can either use natural logs (ln) or logs with base 10. The graphs below show the histogram of population data after a natural log transformation is applied and what the scatterplot looks like if you use a natural log transformation on both the population and land area variables.

進行對數轉換的方式，是逐一計算資料集中每個數字的對數值，並使用其結果當做新資料。一般而言，對數轉換對於修改資料分佈的形狀相當有效，在視覺化呈現時，位於極外側的異常值會被拉到接近其他資料點的位置，讓整張圖看來不會扯得那麼開。做自然對數或以 10 為底數都可以。以下兩張圖顯示了經過自然對數轉換之後的人口資料直方圖，以及人口土地資料的散布圖。

平方根轉換 Square Root Transform

The square root transformation uses the square root of each value instead of the log, and has a more moderate effect on the distribution shape. The two graphs below show the histogram of population data and the scatterplot of population by land area, both after square root transformation is applied .

所謂平方根轉換，就是資料集中數字逐一取平方根，相較於取對數，它對於改變分佈形狀的效果較為溫和一些。以下兩張圖顯示了經過平方根轉換之後的人口資料直方圖，以及人口土地資料的散布圖。

選擇正確的轉換 Choosing the Right Transform

As you develop a better understanding of different transformation methods, you might wonder how to pick between them. The answer to this question is not straightforward and although there are formal statistical methods for selecting a transformation, we often need to use trial-and-error combined with knowledge of different transformations. A general strategy is to apply some of the most frequently used transforms such as log, square root, square, reciprocal, and cube root, and then choose the best one after observing the results.

在學習不同資料轉換方法之時，你可能會疑惑於如何選擇正確的方法。但這問題不好回答，雖然有許多制式的統計法則，通常還是需要在熟習轉換法的知識之後，再以嘗試錯誤方法挑選。要說簡單而通用的選擇策略，就是套幾種常用的，例如對數、平方根、平方、倒數、立方，觀察轉換結果之後，再選用結果較好的。

Looking at the transformed histograms above, the log transformed data seems to be a better fit to the normal distribution while the square root transformed data still carries the right skew. In this example, if you’re doing a statistical test that has assumes the data are normally distributed, the log transformation would be a better method to use than the square root transformation.

一瞥前例的直方圖，經過對數轉換的資料，其形狀似乎較接近常態分佈，而以平方根轉換過的資料，還有一些些向右偏斜。以此例而言，若你所選擇的統計測試，假定其資料為常態分佈，則比起平方根轉換，對數轉換是較好的方法。

On the other hand, if your primary purpose in the example above is to visualize the relationship between state population and land area, the square root transformation does a better job of spreading out the data and making it easier to view than the log transformation.

另一方面，如果處理資料的目的是進行視覺化，以前述州人口數及土地面積範例而言，平方根轉換後的資料仍然夠散開，對於呈現的效果比對數轉換更好。

通用轉換方法 Common Transformations

Method
Math Operation
Good for:
Bad for:

Log
ln(x)
log₁₀(x)
Right skewed data
log₁₀(x) is especially good at handling higher order powers of 10 (e.g. 1000, 100000)
Zero values
Negative values

Square root
√x
Right skewed data
Negative values

Square
x²
Left skewed data
Negative values

Cube root
x1/3
Right skewed data
Negative values
Not as effective at normalizing as log transform

Reciprocal
1/x
Making small values bigger and big values smaller
Zero values
Negative values

方法
數學運算
優於
劣於

對數
ln(x)
log₁₀(x)
向右偏斜的資料
log₁₀(x) 對於 10 的高次方數字特別好用 (1000、100000 等等)
零、負數

平方根
√x
向右偏斜的資料
負數

平方
x²
向左偏斜的資料
負數

立方根
x1/3
向右偏斜的資料
負數
不若對數轉換那麼有效果

倒數
1/x
將大值變小、小值變大。
零、負數

關於轉換的注意事項

Caveats about Transformation

Since data transformation methods involve the application of a mathematical function to your data, you need to be careful when reporting or interpreting any insights derived from the transformed data because a transformation changes the unit of the data. For example, when we apply a logarithmic function to a population variable, the unit of measurement becomes the log of the population. When you’re sharing results, your audience may assume that the calculated numbers or visualizations they’re seeing are based on raw data, so if the values have been transformed, you should clearly communicate what transformation was used, as well as what units the data are being displayed in.

由於資料轉換本質上就是對資料套用數學函數，轉換過後的單位也就隨著變了，對於如何解釋轉換後資料的意義，得格外注意。例如，取了人口數的對數之後，單位顯然就不再是人口數了，而是「人口數的對數」。當你分享資料分析成果時，你的讀者可能會假設那些數字或視覺化呈現，都是以原始資料。若那些數值是轉換之後的結果，你應明確地標示所使用的轉換方法，以及視覺化呈現所使用的數值單位。

If you use transformed data to calculate statistical values like means, you should back-transform the final results and report them in their original units. To back-transform, you just do the opposite of the mathematical function you used in the first place. For example, if you did a square root transformation, you would back-transform by squaring your end result.

若你對轉換過的資料取統計函數，像是平均值，那你應將最終的結果進行反向轉換，轉回原單位，再做報告。所謂的反向轉換，就是套用原轉換用數學函數的反函數。舉例來說，若一開始套用了平方根轉換，那麼其反向轉換的方法就是取平方。

You may not see transforms every day, but when you do, it’s helpful to know why they were used and how they affect your data. It’s important to be able to see different parts of the picture when working with data, and transformations give you another tool to help you do just that!

轉換的資料多半不會每天都出現。但當你見到時，通曉轉換方法更能有助於理解其轉換後資料受何影響。在處理資料之時，能於摸索不同部份而後通曉全局，是十分重要的。資料的轉換，則是能讓你通達的重要工具。

</div>

Visualizing Data

資料可視化

你終於備妥、洗淨且處理好所有食材了。小當家，料理的時間到了！當你正在進行一個媒合的動作的時候，有些食材會在你的料理内擔當「挑樑大角」，有些則是成為「背後的那個人」，而有些則無影無蹤。儘管屬性不同，你還是要確保每樣食材融合成為一道完整的料理。

你還需要去思考如何裝盤以強調重點、機能性和適當性。例如，如果你想著墨於蔬菜濃湯裡的蔬菜，你可能會選擇將其切片烹煮而不是徹底煮爛成泥。然後你還要去找湯碗去盛湯呢，因為總不能用盤子喝湯吧？（至少不是露西亞傳統喝茶）

>同樣的，選定適合的圖像表格和呈現手法是基於數據本身和其相關敘述。某些參數可能會在你的視覺化結果中鶴立雞群，某些則可能是在演算過程中消弭殆盡。除此之外，尚有其他元素亦須斟酌 ── 字體、色彩、符號等。

在本章節内，我們會談到可視化設計如何為你的資料背書説故事。我們還會提及一些視覺化設計上的巧思，像是選擇怎樣的資料作可視化、採用適當的配圖，以及作出最後點綴，呈現一個準確且精美的資料演示，讓你的觀衆為之驚艷。

Deciding Which and How Much Data to Illustrate

Chapter 12

第十二章

Deciding Which and How Much Data to Illustrate

決定哪些資料需要呈現

By Margie Henry

Let’s lay some groundwork for successful data presentation. If done thoughtfully, it will go a long way in helping you determine which aspects of your data to visualize and how. We’ll begin with a little brainstorming. You can do this in just a few thoughtful moments alone or working as a team. Your work here is two-fold: define your message and define your intended audience. You can flip this sequence around, but we’ll begin with defining your message.

讓我們為了有效的數據奠定一些基礎。如果(想要)精準完成(這件事情)，將會(花費)很長一段時間以幫助您確定哪個方面的資料(需要被)可視化及如何(可視化)。 (讓)我們從一個簡單的腦力激盪開始吧。經過些許思辯(thoughtful moments)，您可以獨自或在團隊中完成它。您的任務有兩個(方面)：定義您的(期望)訊息以及定義您的預期觀眾。您可以認為決定其順序，但這裡我們將先從定義您的(期望)訊息開始。

Determine Your Message

決定您的(期望)訊息

Before tackling which data to present, take a few minutes to decide what you want to say. Close down your latest social media craze, step back from your computer, and consider the exact message you want to communicate. Ask yourself, “What do I know, what does it mean, and why do I believe it’s important?”

在暫定(tackling)您所想要呈現的資料之前，花幾分鐘決定您想表達(甚麼)。關掉(那些)社群軟體(譯者案:如Facebook,LINE等)，暫時離開(step back)您的電腦，然後考慮您所想溝通的確切訊息。捫心自問“我所知悉的是甚麼，(分別)代表甚麼含義，以及為何(會被)我認為這是重要的?”

Consider a dataset containing observations on different types of caffeinated beverages and the effects of their consumption. Don’t stop at “caffeine affects the body.” You never want to present information that solicits a “well, duh” response. Dig deeper. Be more specific. What do your data say about how caffeine affects the body? Are the effects all good, all bad, or maybe an interesting combination of both? Do the effects change with a person’s age and/or sex? Are some caffeinated beverages better or worse for overall health? Your answer should be concise: short, sweet, and to the point. A statement such as “Coffee has an ability to reduce the risk of certain diseases and ailments when consumed in moderation because it contains key antioxidants.” goes a lot further than our original example. Even better, it establishes a pretty clear focus for our visuals and some common language to use with our audience.

(請)考慮一個資料集(例子)涵蓋了不同類型的(含)咖啡因飲品以及他們的銷售情況。切勿因“咖啡因影響身體(健康)。”(而停止思考)。您絕對不想在經過呈現資訊(後)得到 “咄~,”(原文為:well, duh。有敷衍的涵義)之類的反應。(再)深入探討。更具體(地說)。你的資料有說明咖啡因是如何影響人體的嗎? 這些影響全部都是好的，(還是)全部都是壞的，又或者是兩者有某些趣味的集合?影響會隨者(使用者)的年紀或性別有所不同嗎?對整體健康而言，含咖啡因飲品是較好還是較壞的影響? 您的回答必須是簡潔:簡短、貼心、扼要。例如“適量應用咖啡，可以降低罹患疾病的風險，因為它包含關鍵的抗氧化劑” 比我們先前的例子更進一步。更棒的是，它為我們的視覺化建立了明確的重點，並提供給觀眾共同語言(了解這件事）。

Right about now you should be having a flashback to English 101. That’s because determining your key message is just like writing a good thesis statement. If you can’t summarize your key message in a few concise sentences then you probably need a better grasp of the topic. Sound harsh? Maybe, but not as harsh as presenting information to a crowd of your yawning disinterested peers. Fight the urge to skip this step! If you’re the paper-and-pencil type, go ahead and write your message down! You can use it as a reference throughout your data visualization process.

現在您或許有種回到英文入門的感受。這是因為決定您的關鍵訊息就像是在些一篇好的論文一樣。如果您無法用幾句簡潔扼要的句子概述您的觀點時，(建議)您(還是)好好理解(所選的)主題。聽起來很殘忍吧?或許，但是總比您在報告時，一大群(公正的)同伴正在打哈欠還不殘忍吧。儘全力完成這個步驟吧！如果您是(習慣)用紙筆的話，(那請)繼續寫下您的想法(message)!您可以將其作為整個資料可視化(data visualization)處理的過程

Simply put, your chances of creating a compelling, well-organized visual argument are immeasurably greater if you begin with a clear and focused message.

簡而言之，您創造了引人注意的機會，若您一開始就有明確且聚焦的訊息，則組織良好的視覺化參數將無可限量。

Understand Your Audience

了解你的觀眾

You’ve determined your message. Let’s now consider the importance of understanding your audience. This knowledge will go just as far in helping you determine which and how much of your data to illustrate.

您(已經)決定了你的(期望)訊息。現在，讓我們來仔細思考了解您的觀眾是多麼重要。 到目前為止的這些知識將協助您確認您的資料中哪些是(需要被)圖示說明的。

Take another couple of minutes and ask yourself “what information is most valuable to my audience,” “what role will my visuals play in this dialogue,” and “what action(s) do I want to incite?” Would you spend time explaining algebra to a group of engineers? (The correct answer is no.) What would be the point? The better you know your audience, the better your chances of creating a successful visual presentation.

再花幾分鐘問自己 “甚麼樣的資訊是你的觀眾(覺得)最有價值的,” “在這次的對話中，您的圖示扮演甚麼樣的腳色,” 以及 “您所策畫的目的?” 您會花時間和一群工程師解釋線性代數嗎? (標準答案是'否')那甚麼將會是重點呢?您越了解您的觀眾，您越能創造一個成功的視覺簡報(visual presentation)。

Let’s imagine presenting data on “Environmental Conservation in the New Millennium” in the following scenarios: (1) on a small-scale blog visited mostly by lay environmentalists; (2) in a classroom of high school students; and (3) at a fundraising event for an environmental conservation organization. Would you create and explain your data the same way to each audience? Hopefully not. You should be able to make a few assumptions about what’s most relevant to present even if you’ve never met a single audience member.

讓我們想像一個關於 “新世紀環境保護”的簡報資料包括下列情境: (1) 一群主要是由業餘(lay)環保主義者(所組成)的人來訪問(您的)部落格時; (2) 高中職的課堂上; (3) 某個環保組織的募款活動上。您會以相同的方式針對每種不同的觀眾解釋您的資料嗎?希望(您)不是如此。你即使不認識任何一個觀眾(也)會做出一些相關的假設。

In our first scenario, we can assume visitors are already interested in conservation. They may have spent time doing actual research. A portion are return visitors who may rely on your specific perspective; they might see you as a content area expert. Your site is, most likely, not the only blog on which they rely, but one day it could be their favorite! At minimum, we can assume they’ve stumbled upon your blog intentionally, and not because of issues with autocomplete. In this instance, breadth and depth are key. You can take more time to explore, deconstruct and restructure the data. If the intention of your site is to incite further exploration, you can presents visuals that pose questions or make viewers question their own beliefs.

在我們的第一個場景中，我們可以假設瀏覽者對環保感興趣。 他們可以花了一些時間做了實際研究。一部分的人可能憑藉著(認同)你的特殊觀點而重新瀏覽(這個blog);他們也許會將您視為這個領域的專家。你的網站(指部落格)最後可能是，但並非唯一他們所認同的部落格。但是在某天(你的部落格)會成為他們的最愛! 至少，我們可以假設他們(花了一些心思)無意中發現您的部落客，但並非自動提醒的情況。在這種情況下，廣度(breadth)和深度(depth)是關鍵(因素)。你可以花更多時間去探討，解構和重構這些資料。如果你的網站目的是策劃更進一步的探討，你可以以視覺化呈現(一些資料)以發表(您的)見解或讓其質疑自己所相信的。

Our high school student scenario is a bit different. You can assume that your audience possesses very little familiarity with the topic. (Though, as always, some members will know more than others.) Attendance may be mandatory, not voluntary: keeping their interest will be key. You’ll want to present fascinating, high-level, attention-grabbing visuals, that address immediate and pressing issues. Approach your vocabulary carefully: explain less-common terminology, and include more visual indicators of good/bad, positive/negative. Your visual display is intended to clearly present the importance of conservation, leaving little room for doubt.

我們的高中同學場景會有一些不同。你可以假設你的觀眾微乎其微熟悉這個話題(雖然，一如既往，有一些同學會了解的比別人多。) 也許(他們都是)被迫出席的，並非志願(出席):(那麼)保持他們的興趣將會是關鍵點。您想呈現迷人的，高層次，吸睛的視覺，這是個是刻不容緩的議題。, (請)注意您的用字遣詞:解釋罕見的術語， 並且包括多個好/壞,正面/負面 的視覺指標。您的視覺化(資訊)是為了清楚地呈現環保的重要性，沒有質疑的餘地

At last, we have our fundraiser attendees. This audience needs to feel that environmental conservation is a cause worthy of their monetary support. It will likely be a mixed crowd: interested donors, their disinterested partners (who just came for free food and drinks), field experts, employees, and interns. You can assume they’ll expect a balance of sentiment, the need for urgency, and solid fact. We’ve assumed the crowd is mixed, so you’ll want to use language that is both familiar and easily understood while not appearing condescending. This audience expects to have their interest in the importance of conservation confirmed and your visuals should accommodate this. As with your student group, leave no obvious question unanswered.

最後，我們還有募款活動的與會者。這些觀眾需要一個讓他們覺得環保是值得他們資助的原因。這可能是一個混合的群體：感興趣的投資者，(與其同行)無利益關係的合作夥伴(僅為了免費的食物和飲料與會)，(某)領域專家，員工和實習生。您可以假設他們將期待(達到某種)情感平衡，(是)需要急迫且確實的事實。我們假設這個族群是混合型的，所以你需要使用熟悉且淺顯易懂的語言，同時勿表現居高臨下(的感覺)(譯者案:指自命清高地用生澀的語言),這些觀眾希望他們所認為的環保重要性(課題)是被認同(or確認)的，您的視覺效果應該要確保這一點。作為您的學生族群，是無庸置疑的。

Presenting emotion-driven content doesn’t mean leaving out key facts if they don’t fit into your ideal storyline. Be extra careful when sharing cause-driven content, and do your best to ensure that your values don’t interfere with an accurate presentation of the data!

呈現感情導向(emotion-driven)的內容並非表示脫離了關鍵的事實，(即使)(這些設定)不適合你所屬意的故事情節。當您分享成因導向(cause-driven)的內容時，請務必小心，請確保當您使用精準的資料呈現時，您的價值觀(values)不受干擾。

Now that we’ve discussed the importance of determining a key message and understanding its audience, let’s delve into deciding which data to illustrate.

現在我們已經討論了決定關鍵訊息以及了解相對應的觀眾的重要性，讓我們更深入探討哪些資料該被呈現。

Deciding Which Data to Illustrate

決定哪些資料(是需要被)製圖

You can begin the process by expanding your key message into a narrative or story. Our goal is to present a sequence or set of facts which gradually leads your audience to the key message. The data you choose to illustrate should set the context, establish the main points of interest, and explain how these are interconnected. Be intentional in what you present, but do not censor data to further your argument. Your visual story should be based on what the data—and not only what you want to—say.

您可以開始進行這階段的工作,(藉由)將您的關鍵訊息擴展成敘述或故事(開始)。我們的目標是呈現一系列或遺族事實以逐漸引導你的觀眾(了解您的)關鍵訊息。你所選擇的呈現資料應(具)上下文關聯，建立(讓人感興趣的)主要的(觀)點，並解釋他們是如何互相關聯的。有意圖地呈現你(所想呈現)的(論點)，但切勿過度解釋(你的論點)。你(所呈現的)視覺故事應該是基於資料而不僅僅是您想 說甚麼而已

Take, for example, the following table presenting the I.Q. scores of children who were adopted at a young age and the socioeconomic status (based on income and occupation) of both their adoptive and birth parents. These data are taken from C. Capron and M. Duyme’s 1989 study, “Children’s IQs and SES of Biological and Adoptive Parents in a Balanced Cross-Fostering Study,” published in the European Bulletin of Cognitive Psychology.

舉例來說，下表描述了兒童時期被收養的孩童智商(I.Q)與養父母、生父母社會經濟地位(socioeconomic status)(譯者案:簡稱社經地位，英文縮寫SES)(指基於收入和職業的評量)(的關係)。這些參考自 C. Capron 和 M. Duyme’s 在1989的研究, “Children’s IQs and SES of Biological and Adoptive Parents in a Balanced Cross-Fostering Study,” 發表於 European Bulletin of Cognitive Psychology。

I.Q. | 養父母社會經濟地位(SES) | 生父母社會經濟地位(SES)
136 | 高 | 高
99 | 高 | 高
121 | 高 | 高
133 | 高 | 高
125 | 高 | 高
131 | 高 | 高
103 | 高 | 高
115 | 高 | 高
116 | 高 | 高
117 | 高 | 高
94 | 高 | 低
103 | 高 | 低
99 | 高 | 低
125 | 高 | 低
111 | 高 | 低
93 | 高 | 低
101 | 高 | 低
94 | 高 | 低
125 | 高 | 低
91 | 高 | 低
98 | 低 | 高
99 | 低 | 高
91 | 低 | 高
124 | 低 | 高
100 | 低 | 高
116 | 低 | 高
113 | 低 | 高
119 | 低 | 高
92 | 低 | 低
91 | 低 | 低
98 | 低 | 低
83 | 低 | 低
99 | 低 | 低
68 | 低 | 低
76 | 低 | 低
115 | 低 | 低
86 | 低 | 低
116 | 低 | 低

Let’s discuss two possible narratives that you could create from this dataset: “Children’s Intelligence Limited by Adoptive Parents’ SES,” and “Adopted Children’s Intelligence Influenced by Both Biological And Adoptive Parents’ SES”.

我們來討論(藉由)這些資料集可以得到的兩個可能的敘述:“兒童的智力受限於養父母的社會經濟地位(SES)”和“受領養兒童的智力受生父母與養父母的社會經濟地位(SES)影響”。

Children’s Intelligence Limited by Adoptive Parents’ SES

兒童的智力受限於養父母的社會經濟地位(SES)

We can create a story supporting the first message by solely looking at the adoptive parents’ socioeconomic status: children of those adoptive families with a high SES had a mean I.Q. of nearly 112 whereas those adopted by a low SES family had a mean I.Q. of 99. But, this narrative would only include half of the relevant information: it leaves out entirely the SES of the child’s biological parents. Understandably, this could play just as big a role as the family’s socioeconomic status would likely impact the level and quality of prenatal care, and, in turn, the in utero development of the child.

我們可以創立一個故事以支持第一個論點--僅藉由觀看養父母的社會經濟地位(SES):這些被收養的孩子(因)養父母擁有較高的社經地位(SES)而平均智商接近於112，而養父母(若)較低的社經地位(SES)則(孩子的)平均智商為99。但是，這樣的陳訴只包括了一半相關的訊息(量):(這)完全忽略了孩子的生父母。可以理解的是，家庭的社會經濟地位對於產前胎教的水準與質量扮演(者)很大的角色，(進一步低說,)(影響到)在母體中的胎兒。

Adopted Children’s Intelligence Influenced by Both Biological And Adoptive Parents’ SES

受領養兒童的智力受生父母與養父母的社會經濟地位(SES)影響

A little more boring of a title, but far more accurate. When we include both the adoptive and biological parents’ SES we get a much better picture of the impact that each has on the child’s I.Q. Specifically, we see:

(這)一個無聊的標題，但是非常精準。當我們(同時考慮)養父母與生父母的社經地位(SES)時，我們取得更淺顯易懂針對每種兒童(的資訊)，(讓)我們看看(這張圖):

![IQ高低分配](../images/sections/05/adoptive-table.png)

So, more correctly, a child’s I.Q. is a function of both his or her biological and adoptive parents’ socioeconomic status. If both have a high SES, the child’s I.Q. will tend to be the highest. If one has a high SES and the other a low SES (it doesn’t matter which set of parents has which), the child will typically have an average I.Q. And finally, if both have a low SES, the child will tend to have a below-average I.Q.

所以，更確切地說，孩子的智商(IQ)是他們的生父母與養父母社經地位(SES)的函數。如果兩者(都具)較高的社經地位，則孩子的智商將高於平均值。如果其中一方較高而另外一方較低(無論那邊的父母(都一樣))，孩子通常是一個平均智商。最後，如果兩者都是較低的社經地位(SES)，則孩子(的智商)將將低於平均值。

Our first example is a clear illustration of what happens when you create a story based on what you want to say, and not what the data say. Unfortunately, applications of data such as this are neither uncommon nor farfetched. We see this done on the news and during casual conversation. The omission of key facts and related variables creates a visual that is full of misinformation. It lacks credibility and presents obvious biases. The second instance presents far less outright bias, is a plausible story based on the data available, presents context, introduces all variables, and explains how the variables are connected. Although it will usually result in a less-sensationalized title, a full presentation of all relevant data is the only way to maintain a credible and airtight argument.

我們的第一例子非常清楚地呈現發生了甚麼事情--當你創立了一個故事(是)基於你想說的(論點),而不是數據想說的(論點)。 很遺憾的是，資料的運用(就像這樣)即不罕見也不(如此)牽強附會。我們可以在新聞和閒談中(看到這些現象)。關鍵事實與相關變化因素(variables)的遺漏創建了一個充滿錯誤訊息的視覺(資料)。它缺乏公性力(credibility)並存在明顯的偏見(biases)。第二個例子則遠遠少了明顯的偏見，是一個基於可用數據的合理故事，(所)呈現的內容介紹了所有的變化因素(variables)，並解釋這些變化因素(variables)是如何被連結的。雖然它的結果往往是一個不太煽情的標題，(將)所有相關資料(做成)一個完整的報告是唯一一種方法(可以)維持可信度(credible)和嚴謹論點(airtight argument)

Deciding How Much Data to Illustrate

決定哪些資料(被)呈現

In previous sections we’ve gone over how to determine a key message, the importance of identifying the audience, and a process for isolating facts to illustrate. We can work on determining how much of our data we need to visualize.

在前面的章節中，中我們已經討論了如何確定關鍵訊息，重點在於分辨觀眾(的不同)以及(將完全)不同的事實轉換為圖示說明。我們可以(進行)決定多少資料需要轉換成視覺呈現的工作(了)。

If illustrating data is supposed to make information more digestible, then care should be taken not to present more than the audience expects, or more than they need to be able to understand your message. As you decide how much data to illustrate, keep in mind the idea that more is not always synonymous with better unless it’s meaningful and presented in support of your key message. In most instances, your visuals will be taken as part of a narrative, contents in a storehouse, or maybe a combination of both.

如果資料圖示應該讓資訊更易理解，那麼(就要)注意應採取不要過度解釋(而)超過觀眾的期望，或(你的訊息)超出他們所能理解(的範圍)。當你決定要(使用)多少的圖示(來呈現)，(請)記住這個觀念: 多 並非總是 好 的同義詞，除非它是有意義的，並且支撐著你的關鍵訊息。在大多數的案例中，你(所想呈現)的視覺效果應採取一段敘述、智庫內容(所取代)，或者結合兩者。

As previously discussed, a narrative is a simply a story presenting a sequence of facts which gradually lead your audience to the key message. When you think of the narrative, think of written reports, PowerPoint presentations, and individual articles in newspapers and magazines or online. You want to illustrate just enough data for your audience to easily identify and understand your perspective without becoming exhausted. Each illustration should have a specific purpose. Avoid including visuals simply because they impress. As a test, try removing one or more illustrations or rearranging the presentation order. Does your narrative still make sense? Each illustration should stand alone, without too much verbal or written explanation, but if it doesn’t add to the audience’s understanding, it’s probably not needed.

如前文(所)討論，敘事(narrative)是一種(將)序列化的事實呈現為簡單的故事並藉此逐漸引領你的聽眾(了解)你的關鍵訊息。當你考慮(想用)敘事(narrative)(呈現時),(請)想想(or參考)書面報告，PPT簡報，報章雜誌或者網路上的個別文章。你為想(達到)不需要精疲力盡(就)可以讓你的觀眾輕易辨識和了解你的觀點(而)使用充裕的圖檔(illustrate)。每一個圖檔(illustration)應該有一個特定的目的(purpose)。避免包含簡易的視覺因為(為了使得)他們留下深刻映像。作個實驗，嘗試刪除一個或者數個圖檔(illustrations)或重新編排您報告的順序。你的敘事(narrative)仍然有意義嗎? 每個圖檔(illustration)應該各自獨立，不需要太多口頭或書面的解釋，但是如果它沒有增加你的觀眾了解(你的論點)，(那麼)它可能是不需要的(圖檔)。

For the audience members wanting more, you can always provide links or references to additional takes on your data along with detailed commentary to contextualize and further explain the information. If you’d like to impress a technically savvy audience, a graphical appendix could be even be shared as a GitHub repository or a gallery of code gists hosted on bl.ocks.org.

針對(那些)意猶未盡的觀眾，你可以隨時提供額外的連結或參考資料在你的資料上，連同詳細解說整個脈絡與進一步解釋資訊。如果你想給一個精通技術的觀眾留下深刻印象，(可以參考)一些被分享在GitHub的圖像附錄或參考位於bl.ocks.org上的代碼範例。

A storehouse, on the other hand, can be thought of as an information repository. Usually consisting of multiple narratives and stand-alone content, this is an example of when more can be better. Unlike those of a narrative, storehouse visitors are less susceptible to data fatigue. They respond well to large quantities of data because they expect to spend time building or enhancing their understanding of a topic. The storehouse doesn’t need to focus on presenting a single message. Its audience seeks new treatments of data, a diversity of perspectives, and various dissections of a topic or content area. In the storehouse scenario, the main criterion for deciding how much data to illustrate should be whether something will create redundancy. If your illustration fails to add something new to the mix or to expand on a topic, it can likely be omitted.

智庫(storehouse)，在另一個方面，可以被當作是一個訊息的倉庫，通常由多個敘事(narratives)和獨立的內容(所組成),這是一個越多越好的例子。不像那些單一的敘事(narrative)，訪問智庫(storehouse)不太容易感受到資料疲乏(data fatigue)。他們對於大量量體(quantities)的資料有較好的反應(respond)因為他們預計花費時間去建置或增強他們理解這個議題(topic)。智庫(storehouse)不需要在單一的訊息上聚焦。它的觀眾在尋覓資料的新解法(treatments)，觀點的多樣性(diversity)，以及一個主題或區域內容的各種剖析(dissections)。在智庫(storehouse)的場景中，決定需要圖像化(illustrate)的資料多寡的主要標準是創立這些東西是否(將造成)冗餘性(redundancy)。如果你的圖檔(illustration)無法新增一些新的(效果)以混合或解釋所談的話題，(那麼)它可能需要被忽略。

To exemplify, let’s imagine a cinephile and store manager. Both are browsing a blog filled with upcoming movie release dates, reviews, and critiques of directors. The cinephile spends hours on the site, soaking up each and every visual and reading through its content. The manager simply wants to know what popular movies he should order for the next holiday season. The manager probably wouldn’t want to spend hours trying to find his answer. For our cinephile, more is better; for the manager, less is more.

舉例說明，讓我們想像一個電影愛好者(cinephile)和一個店經理(store manager)。同時瀏覽一個充滿即將上映電影的上映日期，影評以及導演評論的部落格(blog)。電影愛好者(cinephile)花了數小時在這個網站上。浸淫每一個視覺內容並讀取其內容。(而)店經理只想知道那部是熱門電影好讓他在下個假期(前)訂購。店長大概不會想花數小時(來)試圖找到他想要的答案。(所以)對電影愛好者(cinephile)(而言)，(資料)越多越好，對店長(manager)(而言),少即是多。

Editing and Revising

編輯與矯正

Here’s a frequent and frustrating occurrence: you did your brainstorming, made a bunch of visualizations, and edited down to the best subset to include in your project. You were careful not to overwhelm your audience and you made sure that your illustrations covered the most important key points without being redundant.

How maddening, then, to field questions in a presentation, or see comments in a story or blog post, calling for the very visualizations that you left on the cutting room floor! You second-guess your calls, resist the urge to argue with the person asking the question, grit your teeth and grumble.

It’s okay. If you do a good job and engage your audience, they will naturally be curious and want more information. They might want to see the same data presented in a different way, to dig down, or to zoom out. If these questions mirror the decisions you were making in your selection process, that’s good news! It means you are on the same wavelength as your audience, and that they are involved and interested in the story your data tell.

There are several ways to keep (but de-emphasize) the visualizations that did not make the cut in your main collection. For slideshows, it is common practice to have a collection of extra slides after the “thank you” or conclusion slide that contain information that might be interesting but that won’t fit within the time limit. “Yes, I do have that broken down by [industry sector/year/country/gender],” you say confidently as you flip to the prepared slide. Voila!

Another way to do this would be to publish interactive versions of your visualizations that allow the viewers to dive in and explore the information themselves. If you’re able to share the raw datasets, that’s even better! That way, those who wish to dig deeper and understand the data in new ways will have the option to do so. We’ll talk more about static and interactive graphics later in the Print vs. Web chapter.

If you’re looking for early feedback and you’re not exactly sure where to turn, you can check out HelpMeViz, a community site where you can post your works-in-progress and receive friendly suggestions on how to improve. Getting feedback from your audience and revising your visuals to better fit their needs is all a part of the process!

如果您正在搜尋較早的回饋且您無從下手時，您可以查閱HelpMeViz, 這個社群網站可以發表您的工作進度，並且可以獲得友善的改進建議。 收集您的觀眾的回饋並依據他們的需求改進，這將會是這個過程的所有環節。 </div>

Graphing the Results of Checkbox Responses

Chapter 13

Graphing the Results of Checkbox Responses

By Ellen Cooper

This chapter focuses on checkbox responses or multiple response questions, where a question can be answered with more than one answer, if applicable.

Checkboxes Vs. Radio Buttons

Let’s say you’re doing a survey and you’re interested in what multimedia devices your respondents have used over the last six months. You would use a checkbox response question if you wanted to find out all of the multiple devices that people used over the six-month period. A radio button only allows respondents to select a single answer, so you could only use it to find out, for example, which one device a person used most often during that same six-month period. Each type of question has merit; which you should use just depends on the purpose of your question and how you are going to use the results.

What a Checkbox Question Really Is

So here’s the most important thing to know about checkbox questions, and it’s why you have to consider how you graph the results of checkbox questions differently than you do the results of other types of questions. Checkbox questions aren’t really their own question type! They’re actually just a shorthand way to write a series of yes/no questions. A respondent checks a box if an answer choice applies and leaves it blank if it doesn’t.

We have the checkbox format because it makes surveys more streamlined and easier to understand. In the example below, we asked, “Which of the following electronic devices have you used in the past 6 months? Please select all that apply.” The premise behind the question is that it’s likely that a respondent could use more than one electronic device over a 6-month period, such as a cell phone and a tablet.

If we were to pose this as a series of yes/no questions, it would read something like this:

In the last 6 months, have you used a/an:
Desktop PC? | Y / N
Desktop Mac? | Y / N
iPad? | Y / N
Tablet (other than an iPad)? | Y / N
Laptop (Mac or PC)? | Y / N
Cell phone? | Y / N

With the checkbox question, survey respondents only need to check the answers that apply to them, while in a series of yes/no questions, they would need to respond to every question, even if all their answers were “No”. With a checkbox question, you can simply provide a “None” option at the bottom of your choice list to handle this situation. When several yes/no questions are related, checkbox questions also prevent repetition of instructions, since all the questions are grouped into one.

These changes can help improve survey readability, flow, length, and overall response rates. However, if you want to handle the resulting data correctly, it is very important for you to remember that the underlying structure of a checkbox is actually a series of dichotomous questions.

How Checkbox Answers are Received

How your results or raw data are compiled will, of course, depend on the program you are using to design and distribute your survey. One of the more common formats is shown in the table below; this particular data structure reflects how a checkbox question serves as a quick way to represent a series of yes/no questions. A “1” is shown when a device was selected and a “0” if a device was not selected.

Date | Q1_PC | Q1_Mac | Q1_Tablet | Q1_iPad | Q1_Laptop | Q1_Cellphone | Q1_None
10/02/2013 | 1 | 0 | 1 | 0 | 0 | 1 | 0
10/01/2013 | 0 | 1 | 0 | 1 | 1 | 1 | 0
09/30/2013 | 1 | 0 | 0 | 1 | 0 | 1 | 0
09/30/2013 | 1 | 0 | 1 | 0 | 0 | 1 | 0
09/30/2013 | 0 | 1 | 0 | 0 | 1 | 1 | 0
09/30/2013 | 0 | 1 | 1 | 0 | 0 | 1 | 0
09/30/2013 | 1 | 0 | 0 | 0 | 1 | 1 | 0
09/27/2013 | 1 | 0 | 0 | 0 | 1 | 1 | 0
09/26/2013 | 1 | 0 | 0 | 0 | 1 | 1 | 0
09/26/2013 | 0 | 0 | 0 | 0 | 0 | 0 | 1
| 6 | 3 | 3 | 2 | 5 | 9 | 1

You might also receive results like this:

Date | Response
10/02/2013 | PC, Tablet, Cellphone
10/01/2013 | Mac, iPad, Tablet, Cellphone
09/30/2013 | PC, iPad, Cellphone
09/30/2013 | PC, Tablet, Cellphone
09/30/2013 | Mac, Laptop, Cellphone
09/30/2013 | Mac, Tablet, Cellphone
09/30/2013 | PC, Laptop, Cellphone
09/27/2013 | PC, Laptop, Cellphone
09/26/2013 | PC, Laptop, Cellphone
09/26/2013 | None

Or like this:

Date | Q1_PC | Q1_Mac | Q1_Tablet | Q1_iPad | Q1_Laptop | Q1_Cellphone | Q1_None
10/02/2013 | Q1_PC |  | Q1_Tablet |  |  | Q1_Cellphone |
10/01/2013 |  | Q1_Mac |  | Q1_iPad | Q1_Laptop | Q1_Cellphone |
09/30/2013 | Q1_PC |  |  | Q1_iPad |  | Q1_Cellphone |
09/30/2013 | Q1_PC |  | Q1_Tablet |  |  | Q1_Cellphone |
09/30/2013 |  | Q1_Mac |  |  | Q1_Laptop | Q1_Cellphone |
09/30/2013 |  | Q1_Mac | Q1_Tablet |  |  | Q1_Cellphone |
09/30/2013 | Q1_PC |  |  |  | Q1_Laptop | Q1_Cellphone |
09/27/2013 | Q1_PC |  |  |  | Q1_Laptop | Q1_Cellphone |
09/26/2013 | Q1_PC |  |  |  | Q1_Laptop | Q1_Cellphone |
09/26/2013 |  |  |  |  |  |  | Q1_None

All three of the above examples represent the same answers, but they’re formatted in different ways. Since different survey collection tools format checkbox responses in different ways, you may need to reformat your data to match the specific format required by the visualization software you are using.

Let’s take a look at a summary of possible responses to the checkbox question posed above.

Table 1 Electronic Devices Used | Total
PC | 421 (84%)
Mac (desktop) | 300 (60%)
Tablet (any kind) | 285 (57%)
iPad | 185 (37%)
Laptop | 200 (40%)
Cell phone (any kind) | 450 (90%)

You may notice that the total number of responses (1,841) is greater than the number of people that did the survey (N=500)! Why? It’s because of the whole “a checkbox is really a bunch of yes/no questions rolled into one” thing. The total possible number of checked boxes in a checkbox question? It’s the (# of “real” answer options) X (# of respondents). (Here, a “real” answer option means one that isn’t “None,” “N/A” or “Prefer not to Answer,” since selecting one of those options would prevent a person from choosing any other answers in additional to that.) For this survey, there were 6 device options (aside from “None”) that a person could select and there were 500 people that answered the survey. So the total number of boxes that had the potential to get checked during the survey was 3000, not just 500.

Displaying Your Results

Since the total number of responses is greater than the number of respondents, you need to use some caution when creating graphs based on these data. There are a few different ways to illustrate your results, depending on what your overall question of interest is.

Bar Charts

One way is to construct a bar graph and base the percentages on the number of respondents that selected each answer choice, like in the graph below. Clearly, cellphones (90%) and PCs (84%) are the most commonly-cited electronic devices used in the past six months.

However, the fact that the total adds to more than 100% can be unsettling for some. An alternative way to illustrate the results is to base the percentages on the total mentions (1,841). Analyzing results based on mentions is useful when you want the percentages to total 100%.

Keep in mind that this way of displaying data is based on the number of mentions of a device, not the number of consumers who use that device. While you may be tempted to say, “24% of consumers used a cellphone in the past six months,” the bar chart above isn’t actually displaying that information.

Rather than the number of respondents (500), the chart shows the number of responses (1,841) to our survey on electronic devices. So, it would be better to say, “Based on all devices mentioned, cellphones were mentioned approximately one-quarter (24%) of the time, followed closely by PCs (23%).” This percentage represents the number of cellphone mentions (450) out of the total mentions (1,841) and accurately reflects the information on display.

Depending on your question of interest, you can also group your data. Maybe you’re more interested in reporting how many devices people used rather than exactly what devices were. You could make a column chart like the one below.

Warning about pie charts and checkbox questions

Don’t use pie charts if you’re basing your percentages on the number of respondents that selected each answer choice! Pie charts are used to represent part-to-whole relationships and the total percentage of all the groups has to equal 100%. Since the possible sum of the percentages is greater than 100% when you base these calculations on the number of respondents, pie charts are an incorrect choice for displaying these results.

Over Time

If your checkbox responses have been collected over a period of time, say 2009–2013, you could display the responses in a line chart as shown below.

Attitudinal Measurements

So far, we’ve been looking at the use of checkbox questions to gather data on basic counts (e.g. electronic devices used). Checkbox responses can also be used to assess agreement with attitudinal statements. The graph below shows attitudes of homeowners towards owning their home. In this case, since the statistic of interest is what percentage of homeowners agree with each statement, it is probably best to keep the graph as is, with the total exceeding 100%.

</div>

Anatomy of a Graphic

第14章

圖表解析

By Amanda Makulec

當你完成草圖後，下一步強化圖表效果的方式，是確保圖表中的元素已被妥當地標示。圖表中的文字元素能給讀者線索，幫助你的資料說故事，並且讓你的圖表能夠獨立出所屬的文章，保有它的獨立敘事性。

讓我們先釐清一些名詞。當我們在說資料標籤、類別標籤、軸標籤，和其它圖表上的文字時，我們指的是如圖一的文字標籤。

比較有加標籤與沒加標籤的視覺圖表，是說明文字標籤重要性最有效的方式。下面兩個圖表呈現了各國家在2014年索契冬季奧運的得獎數。

以上兩個圖表呈現相同的數據資料(可以看見地主國俄羅斯的獎牌數遠勝過其它國家)。第一張圖中有Excel試算表預設產生的文字標籤，但只提供讀者有限的資訊，而第二張圖則聰明地加上簡單的數字標籤，有效地幫助資料說故事。

在第二張圖中，簡單地增加不同的文字線索能幫助讀者了解圖表內容。看看圖中的主標題、軸標籤、圖例與資料標籤，每一項都輔助讀者理解你想透過圖表傳遞的訊息。

文字標籤可以幫助你說故事，但是不可忽視它佔據了圖表中重要的空間，因此標籤應該儘可能地簡單且直接了當。過份標註軸標籤與資料點，或是太長的主標題，會使你的視覺圖表雜亂並降低它的影響力。相反地，簡潔優雅的圖表往往最能帶給讀者有力的一擊。在可以使用電子工具製作圖表的現在，修改圖表僅需一個點擊或鍵入幾個指令，所以不要害怕實驗你的標籤，來幫助你取得圖與文字的平衡。試著加入、編輯或刪減各種標籤，直到你找到最好的平衡來有效傳達你的訊息。

在多數的情況下，你會希望你的圖表可以獨立在任何的敘述文字外。有時候，這需要附上額外的說明文字來清楚地表達你的訊息，這個說明可能會附在圖表之上或之下，以便將訊息傳達給特定的讀者。為了簡便起見，本章將著重討論圖表中關鍵的標籤和圖例，而非額外的說明文字。

清楚且適當的坐標軸

首先讓我們釐清幾個名詞以避免用詞上的混淆。類別標籤category labels指軸線旁用來辨別各個資料的標識和分類。坐標軸標題指用來說明該坐標軸使用何種變量的文字標題。如果有需要，請參考下圖一。

你應該用軸標題axis titles來摘要說明你使用的變量。根據你所呈現的資料，你的變量可能非常冗長且敘述性(尤其當你的資料來自他人並且加入了許多條件資訊)；或者相反地，是過份簡短的縮寫(通常發生在電腦可讀的資料上)。你應該在橫坐標軸與縱坐標軸上寫明標題，並且加上你使用的單位(百分比、美元等等)。軸標題讓讀者了解你提供的資訊為何，並且幫助減少單位上的誤解。

標記類別和坐標軸

在提供讀者資訊與保持圖表的簡要之間找到平衡是非常重要的。你需要確保圖表已提供讀者足夠的訊息來理解資料，同時也保持文字標籤的簡明扼要，使你的圖表不會雜亂無章。

在上述奧運的例子中，橫坐標代表2014索契冬季奧運各國的得獎數。製作圖表時，反覆嘗試幾種軸標籤和主標題的敘述是很正常的。在選定一個軸標題後，你應該試著思考讀者會如何解讀這個標籤。比方說，如果軸標題是「獎牌數」，看起來雖然直接了當，但具體指的是什麼？贏得的獎牌數？輸掉的獎牌數？2014年的得獎數？在歷屆冬季奧運的得獎數？你的主標題應該可以釐清這些問題，而軸標題也可以提供你說故事的機會。

讓我們用「獲得奧運獎牌總數(2014)」來取代「獎牌數」這個標籤。標籤中加上「總數」，讓讀者知道你在談論所有獎牌，而非單一種(金、銀，或銅牌)或是單一項賽事。「奧運」讓讀者知道該圖只有涵蓋奧運的得獎數，並不包括同年的殘障奧運。最後，標籤中的「2014」提醒讀者這些獎牌獲得的年份。除了更改軸標籤，你也可以將這些訊息放在圖表的主標題中，本章稍後將討論這個部分。

單位

當我們說單位時，我們指的是諸如「百分比」、「美元」、「百萬」，或是其它能讓讀者知道你在呈現什麼訊息的描述。如果你使用百分比呈現而非個別計數，或是使用乘數(比方用百萬美元而非美元)，寫明單位將會非常重要。

在這張各國人口數的圖表中，「按百萬計」(in millions)一詞非常重要，但卻很容易被忘記。千萬不要讓這個疏忽發生在你身上！人口資料來自http://www.census.gov/popclock/

標明單位是非常重要的。以下是快速檢查軸標題的要點：

如果你要在一張雙軸圖中呈現兩種以上的資料，確保使用的單位一致。你不該把以月為單位的資料和以年為單位的資料放在同一個坐標軸上！

不要使用簡寫或是首字母縮寫，拼寫出完整的單位(比方使用millimeters，而非mm)。

符號有時能方便地替代文字，但你需要確保你已說明該符號代表什麼。舉例來說，你可以使用「$」來代表「美金」，但這個符號被超過三十個國家用來代表該國貨幣，因此你應該要說明你指的國家為何(例如：US$, A$, Mex$)。

某些時候，為了視覺上的簡潔，當資料類別是一眼即能辨視時，你可以不標註軸標題。在冬奧獎牌數的例圖中，我們在縱坐標上標註了國家，如果你有信心相信你的讀者知道這些是國家，你可以不用另外加註坐標單位為「國家」，以免複雜化圖表。如果你不確定讀者是否能知道這些類別，為了謹慎起見，你應該標註你的坐標軸。

位置與格式

橫坐標軸的標題應該與它的軸標籤平行。使用足夠大的字體方便閱讀，但不要過大到搶占了圖表的重點。

縱坐標軸的標題是否要與該軸標籤平行仍有爭議。一方面，使之平行能讓讀者知道這個標題是與該軸標籤相關。如果將文字旋轉90度，通常有足夠的空間書寫軸標題。但是另一方面，人類(譯註：這裡指習慣橫式書寫的語系族群)並不是非常習慣閱讀直式的文字，讀者常需斜眼歪頭來閱讀該圖表。如果你有足夠的空間且軸標題夠短，可以考慮直接橫向書寫而不要旋轉它。下面有兩個例子，呈現縱坐標標題應該避免如何擺放。

<figure><img src="../images/sections/05/y-axis-labels.png" alt="Positioning y-axis labels" /></figure>

個別的類別標籤也應該標示清楚，並確保任何的文字與數字能容易地被閱讀：不應該讓讀者為了看清楚圖表欲傳達的訊息，而需要測試他們脖子的柔軟度。以下是一些製作圖表時，標示軸標籤的技巧：

僅可能避免旋轉文字。縱向的直條圖中經常會旋轉文字，以讓較長的文字標籤可以放入圖中，但這會造成閱讀困難。如果你發現你的橫坐標軸上使用了旋轉且密集的文字標籤時，試著將圖改為橫向的直條圖，以讓標籤更容易閱讀。

簡化你的標籤。即便你的資料原本擁有冗長的文字作為類別標籤，思考如何簡化它們而又不失原本的意義。

盡量少用粗體和斜體字，僅在有特別意義時使用它們。舉例來說，你可以在你所強調的資料上，用粗體標示該類別標籤。

標示你的資料

標示或不標示，這是一個問題。一個資料標籤data label通常是將資料數值與圖中對應的直條柱或點連結。當你希望你的讀者能夠取得每一筆資料的準確數值時，增加資料標籤將是個有效的方式。增加哪些資料標籤取決於你的讀者是誰，和你想藉此圖表說的故事為何。資料標籤可以增加視覺圖表的細節和價值，但也很容易使圖表繁雜並困惑讀者。

雖然資料標籤最常用使用數值，但它並不僅限於數值。在適當的時候，你可以標上文字敘述來強調某些特定的直條柱或點等等。舉例來說，如果你在製作一張複雜的圖表，呈現過去五十年來某一國家所贏得的奧運獎牌數，你也許想在贏得最多獎牌那年，標示出「最多獎牌」，來幫助讀者在眾多資料中，馬上接收到這年的資料，而不僅是原始數值。

當你在決定使用哪些資料標籤來讓圖表更有效時，考慮讀者的觀點。他們是否需要知道該圖上每一筆的準確數值？你在呈現的是一個趨勢嗎？或是你想用圖表幫助讀者更容易地看到高點與低點？根據讀者不同的需要，你可以從以下三種方式選擇一個使用資料標籤的方式。

選項一：標示每一個資料點

標示每一筆資料可以讓讀者輕易地知道每一個資料點的準確數值。這可以讓直條圖或柱狀圖不需要加入格線。資料標籤的顏色和擺放的位置非常重要。在直條圖或餅圖上，將標籤放直條上或圖餅裡，在可以幫助讀者快速識別重要的資訊。在柱狀圖上，你可以把資料標籤放在柱圖裡或柱圖外。放在柱的頂端可以讓圖表更清晰，並且減少讀者誤讀資料的機會。

在這張直條圖中，其中兩個資料標籤因為過大而無法恰當地放在直條裡。讀者第一眼看這張圖時，可能會將兩筆資料誤判為比它們原有的數值還大。因此我們必須小心，標籤的擺放會影響讀者對資料數值的視覺感受。

標示每個資料點可以提供詳細的資訊，但如果你的資料點非常多，這可能會造成圖表雜亂。點圖就是一個好例子，當放上所有資料標籤時，它容易看起來雜亂無章，特別是當資料標籤是文字時(如國家名)，正如下圖四，太多標籤，太多重疊的文字，使得圖表難以閱讀

你必須判斷你的讀者是否需要知道每筆資料的準確數值，或是只需知道趨勢、高點或低點，如果是後者，那麼你可以採取選項二的標示方式。

選項二：只標示你要強調的資料

當你要強調某些資料來與其它資料對比(例如，在許多國家中強調其中一個國家)，或是強調最高或最低的數值時，特別標示這些資料可以幫助你的讀者閱讀。以下針對不同讀者的兩個例子：

如果你的讀者希望知道得獎數前十名國家之間，獎牌數量多寡的範圍，那麼你僅需標示出得獎數的高點與低點。藉由強調高點與低點，你呈現出獎牌數量的範圍。除此之外，美化你的圖表，像是高點與低點使用不同顏色，也可以讓這些數值更能清晰呈現。

如果你的讀者來自某特定國家，因此希望看到該國的獎牌數和前十名國家的比較，那麼你應該強調並標示出該國(例如美國)的資料點。

選項三：不放置資料標籤

製作圖表往往是為了簡化資訊的視覺呈現。因此，如果你不用加入額外的標籤與文字，便可以有效地傳達你的資訊，那麼請不要放置資料標籤。僅在你需要這些細節時使用標籤來輔助。請以不放置標籤為原則，需要時才增加標籤。

因為人腦較難辨視角度和面積，讀者並不容易在視覺上判斷餅圖各部份的數值大小，所以最好在餅圖上增加資料標籤以清楚呈現你的資料。

簡潔而強大的圖例

圖例legend讓讀者知道圖表中的顏色和花紋的意義。當你的圖表中顏色和花紋並沒有特殊意義時，你不需要放圖例(例如前述的直條圖僅表現獎牌數，顏色並無意義)。如果你的圖表中有數種不同的資料，圖例將會是必要的。

比方你現在不想知道總數，而想知道各國家分別獲得金、銀、銅牌的數量，你可以在長條圖用不同顏色代表不同種獎牌。

<figure><img src="../images/sections/05/no-labels.png" alt="Stacked bar without extra labels" /></figure>

在此圖中，每一種顏色代表一種獎牌，並且在上方的圖例中清楚地定義。此時，你需要考慮你的讀者如何看這張圖。如果把圖例放在不明顯的角落，讀者會比較難快速地找到它，但相反的，若把它蓋在圖中央空白的地方(比方直條比較短的所留下的空白區)，則可能讓讀者閱讀圖表時分心。

這張圖為圖例位置的重要性提供了很好的示範。藉由把圖例平行對齊直條圖並放置在上方，讀者可以輕易地將圖例的顏色和資料類別的顏色連結。圖例的擺放是一種藝術，它能夠幫助你的讀者透過視覺來理解你的圖表。

簡潔但有力的標題

終於到了最後！你已經標示了坐標軸，加上(或決定不加)資料標籤，並確認坐標軸上的類別標籤和數值都已經足夠清晰。現在，是用一兩句話來表達你的故事了：你的圖表標題chart title。圖表標題應該簡潔地告訴讀者，你設計這個圖表所要傳達的訊息。標題不需要是完整的句子，並且可以省略複雜的語言和語法。

考慮讀者是誰，以及你的圖表將在哪裡發表，是下標題非常的重要依據：你是否是記者，正在為一篇報導增加圖表？或者你是一位部落客，在撰寫圖文並茂的文章？或者你是一位科學家，正在為有審稿機制的期刊撰寫論文？科學家也許需要在標題中清楚闡明研究變數的關係，而記者則希望藉由潤飾標題來強化他們要傳達的訊息。

最基本的圖表標題應該清楚呈現橫坐標軸的自變數與縱坐標軸的因變數，兩者間的相互關係。舉例來說，我們把上述奧運獎牌的圖表下標為「2014索契冬季奧運，前十名國家的各類獎牌總數」。這個標題告訴讀者這個圖表是關於各國所贏得的獎牌總數(依不同獎牌計)，並限定於2014索契冬季奧運。圖表標題的每個元素都有其價值，並且告訴讀者一些意義。這樣的標題雖然很枯燥乏味，但它並無問題，因為它直接了當並清楚地告訴讀者此圖表的內容。

下標題時，你可以將它視為該圖表欲傳達給讀者的結論。以上述奧運獎牌圖表中強調美國得獎數一圖為例，如果你想強調美國在所有國家中，得獎數排名第二，你可以為該圖表下標為「2014索契冬季奧運，美國獎牌總數排名第二」。這種引導性的標題經常出現在報紙，或是其它述說特定故事的出版品中。圖表標題提供你強化甚至擴大訊息的機會，你應該好好利用它！

尋求反饋

檢視圖表是否清楚傳達你的訊息，最好的方式之一便是讓一個公正的第三方閱讀。你應該請求朋友或同事花五分鐘的時間看一看，並提出誠實的評論，請教他們閱讀圖表時是否有缺少任何訊息的感覺。如果對方提供修正的建議，不要認為他們是針對你個人，畢竟好的圖表經常經歷數次的草圖修改，也往往需要他人的協助。

這本書的貢獻者也成立了一個群體，來為你的視覺圖表提供意見與想法。如果你在尋找能提供誠實評論的第三方，社群網頁HelpMeViz可以幫助你。

這個最後的檢查，幫助你確保你提供了足夠詳細的軸標籤、軸標題、圖表標題和圖例。當你在上面得到有建設性的建議後，回頭進行修訂--這將會是十分值得的工夫，能讓你的圖表有更完美的成果。

</div>

Importance of Color, Font, and Icons

Chapter 15

顏色、字型與圖示的重要性

By Howard Coale

Form (Frequently) Tyrannizes Function

形式(常)凌駕功能

Listening to someone with a good voice sing a few bars of “I Want to Hold Your Hand” pretty much beats out the experience of hearing someone mutter “E=MC²” any day, but that preference isn’t a reflection on the value of the information in those two phrases. Rather, it describes how the experience of information can change the perception of its value. In other words, as information designers, we aspire to be singers, not mumblers. No, that doesn’t mean “change everything to make it pretty”: it means you must first and foremost acknowledge that you are not presenting information, you are designing an experience of information.

在這章裡面，我們將涵蓋一些基本的設計原理，也會包含可以用來影響、聚焦、闡明、強化讀者體驗與資料理解的工具（特別是針對字型、層次(?)、色彩與圖示）。

In this chapter, we’ll cover a few basic design principles and tools that can be used to sway, focus, clarify, enhance, and sharpen your audience’s experience and understanding of data (and specifically fonts, hierarchy, color and icons).

The Myth of Objectivity

客觀性的迷思

Before we dive into design fundamentals, we need to understand just how much design can impact the perception of information. At its heart, data visualization is a form of storytelling, and no story is ever told from an objective point of view. The graphic representation of data has been used just as effectively by dictators and terrorists to justify acts of violence as it has by scientists to report groundbreaking research.

在我們更深入探討設計基礎前，我們必須要了解設計能如何影響人類對資訊的接收。資料視覺化根本上來說也就是一種敘事類型，而「敘事」本身從來就不可能是客觀的。圖像式的資料表現能有效為科學家傳達突破性的科學研究，但也能為獨裁者與恐怖份子將暴力行為解釋成正義。

To that point, a critical rule in data visualization is to realize you’re creating, at some level, a biased point of view. If you don’t, you’ll have little chance of successfully judging the honesty, accuracy, elegance or utility of your solution. Even the most typical chart should seek to captivate and inspire that part of the human mind that yearns for meaning and narrative while in the act of seeking information.

從這角度來看，便有了資料視覺化中相當關鍵的其中一個規則：你必須要了解你的產出其實有一定程度的偏差觀點。若你不了解這點，就很難評斷你的作法是否誠實、精確、優雅、或其實用性。即便是最典型的圖表也應該要試著吸引這部份的人類心靈 － 渴求意義與故事仍同時追求資訊。

You might be a researcher reading this who wants to ask: “I’m making a graph to show how many times a rat turned left or right in a maze. Why does a graph that displays that information necessarily reflect a bias or a narrative perspective?” The answer is that though the data show the rat turned left 35 times and right 22 times, the way the presentation of that data is designed will — usually within one or two seconds — teach the viewer what is important and what is not. Being aware of that fact during the design process can make your graphic representations of data much more effective and puts you in a better position to think through what you want to express before, and while, you design.

正在讀這段文字的你可能是個研究者，想要問這樣的問題：「我在做的圖表顯示了老鼠在迷宮裡左轉或右轉的次數。這樣的圖表也會反映出敘事或是偏差的觀點嗎？」答案在於，透過資料顯示老鼠轉向次數的同時，資料的表現方式通常也在幾秒內就告訴了讀者哪些部份重要。注意這個事實：設計的過程能讓你的資料更有效的傳達給讀者，也能讓你更清楚地思考你先前想表達的概念，然而，同時你也做了設計。

Fonts & Text

字型與文字

Legibility and Readability

可識性與易讀性

When designing the viewer experience for a given set of information, the most important rule of all is to strive for ruthless simplicity in every choice you make. In the “narrative” of the information you are presenting, the style of presentation should never demand attention (that is, the design should complement the data, not the other way around!). That applies to everything from the design of an illustrative representation like a chart to something as seemingly simple as font selection. In the world of typography, the balance between legibility and readability is a decision that a typographer makes when designing a typeface (more commonly known as a font). Not all fonts that are legible (i.e., one character can be easily distinguished from another) are necessarily readable (i.e., designed in such a way that supports and enhances reading so that the reader easily absorbs the text’s meaning without being distracted by the font).

在為資訊集的讀者體驗做設計，最重要的規則即是 － 都盡量選擇最簡潔的做法。你所傳達的資訊的「敘事」部份中的「樣式」永遠都不該成為讀者注意力的點 ( 也就是說，設計與資料應該是互補的 )。這從如圖表的插畫設計到單純如選擇字型都一樣。在排版的世界裡，可識性 與 易讀性 間的平衡

For instance, there are many fonts that are designed specifically to stand out. For example:

These “display typefaces” are good examples of fonts that are legible, but not readable. They were created to emphasize small pieces of text, like a title or heading, or to convey a specific style rather than communicate information quickly and transparently. In general, you should not use display fonts in data visualization because they create additional work for the reader. You can’t afford your viewer struggling for a second, or even half a second, trying to understand what something says. Nor you do you want a specific font skewing the meaning of your data because your audience automatically associates that font with a specific type of content. We’re looking at you, Charlie Brown, when we say no Comic Sans!

This is one way that data visualization and storytelling are similar. If your audience becomes too aware of the style in which a story is told, they may be distracted from the meaning, or the valuable details in the content. The bottom line is that you should use fonts that are legible and strive to present them in a readable fashion.

Fonts for data visualization that do a good job of communicating the message without calling attention to themselves include Helvetica, Arial, and Gill Sans. In general, it’s best to use sans serif fonts for numbers in charts of any sort because serif fonts tend to have more “visual noise” to be interpreted. For example:

Believe it or not, your mind takes a fraction of a second longer to understand the numbers on the right than it does to understand the numbers on the left. Why? Serif fonts have stylistic variations in their endpoints, widths, curves and shapes that adds more for the eye to take in at any given moment.

Since data visualization seeks to speed the process of insight, you want to stay away from anything that slows down the viewer’s understanding, even if by an infinitesimal amount.

Hierarchy

On the subject of speeding up comprehension, one of the designer’s greatest tools is use of hierarchy. Hierarchy refers to the ways in which we weight and organize information so that the audience can more quickly and easily scan and, more importantly, understand what they’re seeing or reading.

Perhaps the most familiar application of hierarchy is the good old-fashioned outline that we all learned in grade school. By breaking the information into titles, headings, supporting information, and details, you create a clear map by which your audience can quickly scan the information, absorb the content, and look for the details they’re interested in.

Data visualizations with a clear hierarchy tell the reader how to process the information and in what order, whether from the most important information to the least, or from the big idea to the supporting details. Hierarchy can also give your viewer the freedom to choose where to enter into a given set of information, which is especially useful when presenting large amounts of data.

Fonts can help establish hierarchy through variations in size (11pt vs. 14pt), weight (bold, medium, regular), and emphasis (italics, capitals). Color, when used selectively and consistently, can also help delineate hierarchy by creating emphasis and guiding the viewer’s eye in a way that facilitates understanding. Other common tools include line breaks, bullet points, numbered lists, indentations, etc. The means and methods to establish hierarchy are endless, but remember that the guiding rule is ruthless simplicity in all things. You don’t want the tools you use to establish hierarchy to distract your audience from the data experience you’re trying to create.

Color

Conceive in Black and White, Then Add Color

To paraphrase the designer Paul Rand, if a design doesn’t make sense in black and white, it will make less sense when color is added. That’s not meant to imply that graphic representations of data should always be black and white. It only means that balance of line and solid, dark and light, layout and structure are the foundation on which any chart or visualization is built and, as such, the design of these elements needs to be able to stand on its own.

The assumption is that you are making visualizations because you want to provide clarity, insight, and context. Color is a tool, but it should never be the starting point. Instead, conceive and plan your charts and visualizations in black and white whenever possible. Then, apply color conservatively with the primary objective of highlighting and enhancing comprehension. Here are three basic tips to keep in mind when assessing how to use color in graphic representations of data.

Don’t Forget Gray

Gray is one of the most important colors in your palette because it offsets the colors around it. In the world of information design, it is critical for modifying, clarifying and balancing almost any color palette, and taming the overall graphic comprehension of charts.

Colors Can Be Used to Express Value

Don’t forget that color can be used as a more conceptual axis to designate value. That said, when using colors to represent a scale, or gradations of value of any sort, look at what you’ve created with all the color/saturation removed. This helps guarantee that the various tints you are using appear as truly different values to the eye.

The Role of White Space in Reading Color

If you were painting, you could merge colors in one seamless quilt. But in information design, every color you use must have a reason for being there and should be distinct. If you are creating a chart in which a very slow gradation or change from one color to another has meaning — as it might if representing climate data, election turnouts, or other continuous variables — then the changing and merging of colors is central to interpreting the data. However, in many cases, colors need to be distinct since they will represent a unique value or attribute, and to help the viewer’s eye read them as distinct and clear, the colors need room to breathe. That’s what white space can do for you.

Icons

A Short History

The first graphical user interface (GUI) was introduced by Xerox in 1981 in the Star workstation, also called the 8010 Information System, but the ideas that inspired it were first explored by one of its chief designers, David Canfield Smith, in his 1975 PhD dissertation titled, Pygmalion: A Creative Programming Environment. A couple years later, Steve Jobs famously “borrowed” the idea from Smith and Xerox for the Macintosh. Only a few decades later, two thirds of the seven billion people on Earth use mobile phones that are largely driven by graphic, icon-based interfaces.

Yet, simply drawn, easy-to-remember pictograms have been used since Neolithic times to represent information that was too complex to explain in a small amount of space. Within the practice of data visualization itself, iconography has been used as far back as the early 15th century, borrowing from other disciplines that sought precision and accuracy in drawings like architecture, military planning, cartography, alchemy, and astronomy.

Use Skepticism & Lean Toward the Literal

The use of icons is now so widespread and commonplace that they are often added without much thought. People feel they should “just be there” in order for an app to feel like an app or a website to feel user-friendly. As a result, we often see them without ever wondering why they were put there in the first place.

Even so, when creating data visualizations, you should regard the use of icons with caution. If you can make something clear in a small space without using an icon, don’t use one. Why? Because icons can make things more obscure, not less. As soon as anyone asks, “What does this icon mean?”, you’ve lost the battle for simplicity. This is true for icons that are used as metaphorical identifiers in infographics or data visualizations, as well as for functional icons that perform a task or reveal more information within the context of an interactive graph or infographic.

It would be very hard to misunderstand what the icons are meant to represent here: one person-shaped icon equals one person. This is a very literal representation, and that is why it is so easily understood. In fact, the more literal and simple an icon is, the better it is. It is a very similar issue to the one faced when selecting fonts: if the viewers spend too much time acclimating to an elaborate presentation, they will often miss the full value and substance of what is actually being presented. Here is another version of the same information:

The exact same data is being conveyed here, but it takes the viewer slightly longer to understand it. Why? Before the viewer can correctly interpret the data, they have to create a complex metaphor in their head. First, they’re being asked to imagine that this icon represents all twenty test subjects, but they’re then also being asked to divide that metaphorical person by another set of criteria. As simple as they may seem, icons are still asking the mind to pretend they represent something else. The more you ask symbols to represent, the less literal and less clear they become. When it comes to using symbols and icons of any sort, stick to one simple meaning each.

You should also make sure that if you use icons in graphs, you properly handle their sizing. A common mistake is to scale both dimensions of an icon, which leads to an incorrect visual comparison. In the image below, the height of the person figure has been tripled to indicate that Group B is three times the value of Group A. However, in order to keep the ratio of the image, the width has also been tripled. This results in the larger figure actually being nine times the size of the smaller one rather than just three.

Stay Cognizant of Future Needs

It is not uncommon for new traders to join an investment bank and have someone walk them through the company’s software on the trading floor, with the assurance: “Don’t worry, you’ll understand what all the icons mean after you’ve been here a few weeks.” A mass proliferation of icons across an interface or data visualization often indicates that options have been added after the overall planning, design and (in the circumstances of interactive data visualization) development has taken place.

Here is a simplified example of the sort of icon-based fold-out palette one frequently finds in financial analysis dashboards that have gone through several versions in which capabilities have been added over time:

One of the greatest challenges in the graphical representation of data is how to ruthlessly simplify even while the complexity of choice for the viewer or user increases. As in the example above, sometimes icons are used to solve this problem; but as you can see, this can create another problem—confusion.

The best way to avoid this is take time during the initial conceptual work on any given data visualization to assess what the extreme requirements might be in future iterations. Ask yourself what options or views might be added in the future? Or, in the circumstance of complex interactive data views, what possible functionality might be introduced that would change the nature of the overall visualization?

These are just a few of the core guidelines for using icons in infographics or complex data visualizations.

</div>

Print Vs. Web, Static Vs. Interactive

Chapter 16

Print Vs. Web, Static Vs. Interactive

By Sarah Rigdon

Whether you’re designing graphics for static environments or for interactive ones, it’s important to address a few key things up front. First of all, what do we mean by static and interactive?

Defining Print Vs. Web; Static Vs. Interactive

You can design static graphics for print, the web, or both. With static graphics, you’ll provide everything you want your viewers to see without requiring any input from them. The most a static image demands from a viewer is to turn a page or scroll or perhaps enlarge an image. Typically, static graphics display information that isn’t subject to change.

You can also design interactive graphics for print or the web or both. Interactive graphics typically provide viewers with the option to dive deeper into the information, often requiring some degree of input. Now, you may be thinking, “Interactive print?” Actually, yes, though of course it’s not as common as interactive graphics on the web. Some examples include:

Interactive projects

Art installations

Flip books/analog animations

Popup books

So, static isn’t a stand-in for print, nor is interactive a stand-in for web.

One more thing: the antonym to static isn’t interactive. It’s changing, moving, or dynamic. Motion graphics don’t require viewer interaction, but they’re not static, either. Any given cable news network will give you examples of these. Do they display necessary information? Not usually. Adding motion to infographics can keep viewers’ attention, at least at first. But over the course of a segment or a show, viewer fatigue sets in. Motion graphics often look impressive, but you should always ask whether motion is necessary for conveying the information and be wary of overusing it.

Now let’s shift gears and talk about some of the core design concepts you should consider when creating static and interactive print-based and web-based graphics.

Color

As for color, there are a lot of questions to take into consideration before you break out your palette, especially if you’re working in print. Are you working with full color or black and white? If you’re working with color, consider what you’re gaining that makes it worth the cost difference from black and white. After all, grayscale and two-tone printing can still communicate a lot of information. Is a full-color print job really necessary?

How many colors are available for the print job? What kind of paper are you working with? Colors work differently with news print, magazine, and printer paper. Or are you getting fancy and putting the graphic on merchandise? A good merchandising company will work with you to make sure your t-shirts, totes, buttons, or stickers look great, but you’ll want to anticipate these concerns in advance.

On the web, a lot of these restrictions go away, but not all. Many designers have to work within a website’s color palette. They have to ask questions such as: does the website’s green color scheme completely clash with this bright pink graphic? Does it subtly complement the branding and logo design? The answers to each of these can have an impact on how the viewer receives the final design.

Size

Step back and take a look at the space you’re working with. You have to consider size for both print and the web.

With print, you have the page size constraining your graphics, but that can also be a blessing because you only need to design for a single width and height.

On the web, you can often include image-zooming functionality or allow viewers to open a larger version of a graphic in a separate browser tab, but you should consider the effects that zooming and scrolling can have on a user’s experience. A well-designed graphic should provide key information in a clear and elegant manner with minimum effort required by the user.

Mobile devices are also a concern, as more people are using cell phones and tablets to browse the web. Keep in mind that nearly two-thirds of cell-phone owners now use their phones to browse the web, and twenty percent use their phones to do most of their web browsing.Cell Internet Use 2013. Pew Research Internet Project. "Main Findings." Duggan and Smith. 16 September 2013: http://www.pewinternet.org/2013/09/16/main-findings-2/ It’s hard to see a lot of detail on a tiny touchscreen, but you don't want users to have to squint to understand your graphic! You could build a responsive graphic — one that automatically changes its layout according to the dimensions of the viewer’s screen size — or you could design a completely different graphic specifically for display on small mobile devices. Given the state of mobile web browsing trends, it’s something you should consider when publishing a visualization on the web.

Many mobile devices (and increasingly, laptop and desktop monitors) have high-resolution displays, often referred to as retina displays. A static image designed at the standard web resolution of 72 dpi (dots per inch) will usually look blurry when viewed on a retina display. You’ll need to make double-resolution retina versions of your image-based graphics if you want them to look sharp on, say, a modern iPad. You can also create them using web fonts and SVGs, and they will always look crisp, no matter what the screen resolution. One downside to this is that some older browsers (notably Internet Explorer 8 and below) do not natively support SVG format.

What's an SVG?

SVG stands for Scalable Vector Graphic: it’s a web-friendly format for displaying vector images—images that can scale to any dimension without pixellating by using mathematical expressions to draw paths. Vector images will look crisp even when blown up to very large sizes, unlike bitmap images (also known as raster graphics), which use bits of specific colors which are mapped to certain regions in the image. Bitmap images will look blurry when viewed at a high resolution, but they are often necessary when the color combinations are complex. A photograph is a great example of a bitmap image, and a simple circle is a great example of an image that can easily be drawn as a vector image.

Many of the tools for building web-based interactive charts and graphs (such as D3.js) use SVGs to render visualizations, especially interactive ones. Most of the static infographics, however, are published in bitmap image formats, such as JPEG or PNG.

Size doesn’t just refer to how big an image appears. Pay attention to the size of your file as well. People all around the world use mobile devices to browse the web. Not only can it take a long time to download a large image across a mobile network, but in some parts of the world, each megabyte of data is an added cost. Be nice to your viewers and work to reduce your final image size.

Layout

With static images, the point is to provide everything you want to convey in one or more frames. There are no mouseovers, you can’t click around and select certain details and un-select certain others (e.g., show only state rodeos from 1988 to 1996). Make sure that your key points are visible. If you’re creating a static version of an interactive graphic, keep in mind that you might not be able to show everything that was accessible in your original interactive graphic. If possible, provide URLs so that your readers can also access the raw data and the interactive version of your graphic. That way, they’ll have the option to dig deeper and find more information about the parts of the data that didn’t make it into the print version.

With interactive graphics, make sure to test any clickable controls that manage the interaction. Are there buttons or slides off to the side, or are they a part of the graphic? Does the viewer have to scroll around to see everything with several options engaged? Test your graphic in different browsers to make sure it all fits—and functions—the way you want.

A Note on Static Graphics on the Web

These aren’t necessarily interactive, save in that viewers can enlarge them. Because the graphic itself doesn’t change, the main considerations of layout still apply. But just by being on the web, static graphics give us more interactivity than print. For example, they can be linked to their data source, another related graph, or a different web page. Mobile screen sizes still remain a challenge for static graphics specifically because their layout doesn’t change to accommodate smaller screen real estates or different width-to-height ratios.

Text labels and visual details that are clear on a larger screen can sometimes be difficult or impossible to read on a mobile device.

Although URLs displayed as part of a static image can’t be directly clicked on, it’s common for thoughtful publishers to paste reference links just below the graphic so that readers can easily access the source data.

Weighing the Need for Interactivity

Making interactive graphics is getting easier all the time with new software and applications. Interactivity can be not only extremely impressive but also useful, highlighting different types of information more effectively than a static image ever could. But sometimes, you’ll come across an interactive image that actually makes the information more confusing. Other times, you’ll come across an interactive image that seems neat at first, but as you play with it, you realize it’s just bells and whistles. You’re not learning anything new, and the information could just as clearly have been presented in a static image. What a let-down!

Why spend time making a graphic interactive when it doesn’t need it? Interactivity is a tool in the designer’s toolbox. Interactivity should be used when it improves upon a static graphic or adds value to the information and does something that the static image alone could not.

For example, consider the following bar chart:

In the figure above, the interactive labels (e.g., the hover effects) aren’t necessary for the viewer’s comprehension. The labels can be added to the chart before hovering, which makes the chart easier to understand and requires less effort from the viewer. When you’re presenting a small, simple chart with a single variable, interactivity can actually detract from your message.

There are plenty of amazing use cases where interactivity adds value to a visualization, but you shouldn’t make yours interactive just because it looks cool. Make sure you’re actually improving the user’s experience. For example, interactivity can help greatly enhance any graphic that illustrates a rich data set you want the user to explore. Let’s say you want to show how different groups of consumers drink coffee. An interactive visualization might allow your user to filter by demographic groups and explore coffee consumption patterns.

Interactive graphics can add a lot of value when you have a rich dataset that you want the user to explore. For example, let’s say that you want to know how different groups of consumers drink coffee. An interactive visualization might allow you to filter by demographic groups and explore coffee consumption patterns.

In static graphics, you can’t rely on visual effects like movement or color changes to make your point. On the other hand, while interactive graphics can incorporate visual effects to show trends across time, they do introduce room for technical difficulties. Be mindful of compatibility across computer systems and browsers. Some of the more complex interactive visualizations require a lot of computational power and can be slow on less-powerful machines such as smart phones. Visualizations made using Flash won’t render on iPhones or iPads, and those made using SVGs won’t work in Internet Explorer 8 or below. Interactive visualizations built for a computer with a mouse may be difficult or impossible to use on touchscreen devices if the creator forgets to test and optimize the graphics for touch-based interactions.

Any medium for sharing a visualization — whether static or interactive — will have its strengths and its limitations. Think about your story, your audience, and how readers are likely to view your visualizations. Ask for feedback and continue to iterate on your process. And if you’re looking for some friendly feedback and suggestions, you can post your works-in-progress on HelpMeViz.

</div>

What Not To Do

禁斷的組合

在料理時有些禁斷的組合是不能夠合體的：

小蘇打加上酸醋

食物未煮熟上桌

已知用火上澆水

太陽餅裡放太陽

慶記外送裝作煮

不要問，很恐怖。

資料視覺化當然也會有禁斷的組合，本章節就是意圖使你遠離邪門歪道。

Perception Deception

Chapter 17

Perception Deception

By Mizel Djukic

With great power comes great responsibility.

So far you’ve picked up a bunch of tools you can use to build compelling stories through visualizations. If the story you tell is powerful enough, it can change people’s minds and move them to action. But before you rush out there to change the world with your newfound powers, let’s go over some of the accidents your visuals might cause because of the quirky way our brains perceive things.

How We Perceive Differences

Our brains love a difference. Even when we’re just relaxing in front of our computers watching cat videos and drinking iced tea, our brains are constantly looking for boundaries in our environment, differentiating between our feet and the floor, our hands and the keyboard, our cup and the table. How we perceive these boundaries depends on a variety of factors, including color, contrast, shape, and perspective.

Differences in Color and Contrast

When it comes to color and contrast, the example that probably comes to mind is black and white. It’s very easy for us to distinguish between the two because one is so dark and the other is so light. Varying the lightness and darkness of a color (known as the value) is one way to establish contrast, but it’s not the only way. You can also vary the saturation (or chroma) so that a color looks pale and faded or bright and vivid. Or you can establish contrast by simply using different colors (or hues) that are easy to tell apart from each other.

How our brains perceive contrast depends on the context. The higher the contrast between objects, the more different we think they are.Cairo, Alberto. The Functional Art. Berkeley: New Riders, 2013. Print. For example, take a look at the following graph:

Anything stand out to you? Probably the red line, for a few reasons. First, the red line is more saturated than the other lines. Second, red and blue are good contrasting colors. Third, to our eyes, reds and yellows come to the foreground while blues recede to the background.Rockwell, Ken. The Secret: What Makes a Great Photo. Web. 1 March 2014. http://www.kenrockwell.com/tech/basics.htm

Because of the high contrast between the red line and everything else, our brains will naturally focus on it, and we’ll probably think that there’s something special about the red line. Having the red line stand out like this is fine if there’s a good reason for doing so. For example, if each line represents the level of happiness over time in different neighborhoods, then the red line can highlight your neighborhood. However, if it turns out that there’s nothing special about the red line, then it winds up being distracting and confusing.

Now take a look at the following map which shows the level of excellence in different parts of a land called Relethounia:

What stands out to you this time? Maybe that there seems to be some sort of division between the western and the eastern parts of the country? This apparent division is the result of the relatively high contrast between the light green and the dark green areas. So it seems like there’s a bigger difference between these two parts than between, for example, the yellow and the light green areas. But when we look at the legend, we see that the light green parts differ from the yellow and the dark green parts by the same amount: one level of excellence.

So be careful with color and contrast. When people see high contrast areas in your graphs, they’ll probably attach some importance to it or think there’s a larger difference, and you want to make sure your graphs meet that expectation.

Differences in Shape

Our brains have a much harder time telling the difference between shapes. To illustrate, see how long it takes you to find the red circle in each of the figures below:

Adapted from: Healey, Christopher G., Kellogg S. Booth, and James T. Ennis. "High-Speed Visual Estimation Using Preattentive Processing." ACM Transactions on Computer-Human Interaction 3.2 (1996): 4.

Did you find the circle faster in the first figure? Most of us do because our brains are better at differentiating between colors than between shapes. This is true even if some other factor is randomly changing and interferes with what we see.Healey, Christopher G., Kellogg S. Booth, and James T. Enns. "High-Speed Visual Estimation Using Preattentive Processing." ACM Transactions on Computer-Human Interaction 3.2 (1996): 107-135. Web. 16 June 2014. For example, see how long it takes you to find the boundary in each of the figures below:

Adapted from: Healey, Christopher G., Kellogg S. Booth, and James T. Ennis. "High-Speed Visual Estimation Using Preattentive Processing." ACM Transactions on Computer-Human Interaction 3.2 (1996): 5.

Most of us find the color boundary (between red and blue) in the left figure more quickly than the shape boundary (between circles and squares) in the right figure, even though both figures have another factor that’s randomly changing and interfering with our ability to detect the boundary. In the left image, the objects are organized by color while the shape is randomly changing. The reverse is true in the right image: the objects are organized by shape while the color is randomly changing.

All of this brings us to the question of whether or not it’s a good idea to use icons or pictograms in our visualizations because the simplest icons are defined by their form, not color. Luckily for us, the chapter on The Importance of Color, Font, and Icons offers some wisdom:

The rule should be: if you can make something clear in a small space without using an icon, don’t use an icon. […] As soon as anyone asks, “What does this icon mean?” you’ve lost the battle to make things ruthlessly simple.

Icons can be a clever and useful way to illustrate an idea while making your infographic more attractive, but in order to be effective, they have to strike the right balance between detail and clarity. Too much detail leads to icons that are hard to distinguish from each other; too little detail leads to icons that are too abstract. Both overcomplicated and oversimplified shapes can lead to a confused audience.

Now you be the judge. Good icons can communicate an idea without the benefit of color. Each of the three maps below shows the locations of hospitals and libraries in an area. Based solely on the icons, which map is the quickest and easiest for you to understand?

Differences in Perspective

We live in a 3-dimensional world with 3D movies, 3D printing, and 3D games. It can be very tempting to add some of this 3-dimensional razzle dazzle to your 2-dimensional visualizations. Most of the time, though, people end up more dazed than dazzled by 3D graphs.

As mentioned earlier, our brains already have a hard enough time distinguishing 2-dimensional shapes from each other. Adding a third dimension to these forms often adds to confusion as well. But don’t take our word for it. Which bar graph do you think is easier to read?

Harder-to-read graphs aren’t the only reason to avoid 3D in your visualizations. The other reason is that our brains associate differences in perspective with differences in size. You’ve probably experienced this phenomenon many times, whether you’re looking down a street or up at a skyscraper: objects in the foreground appear bigger while those in the background appear smaller.

For example, take a simple square table. You know that because it’s a square, all four sides are the same size. But when you look at it from the side:

It seems like the side closer to you in the front is bigger than the side farther from you in the back.

For this reason, many types of 3D visualizations can be misleading. In the pie chart below, which slice do you think is bigger: the red slice or the green slice? the blue slice or the yellow slice?

In terms of which piece represents more data, the red slice and the green slice are actually the same size. As for the blue and the yellow slices, the yellow slice is bigger: in fact, it’s twice as big as the blue slice. This is not clear from the 3D pie chart because of the way perspective distorts the size of the slices, but it is clear when we look at the same pie chart in 2D form:

So unless you have a really, really, really good reason to use 3D graphs, it’s better to avoid them. But just because 3D graphs aren’t great doesn’t mean 3D is bad in general. For example, 3D visualizations can be very useful in some types of scientific modeling, such as a 3-dimensional model of the brain that highlights active regions, or a 3-dimensional map that models the topography of an area. There are definitely instances when 3D enhances a visualization. Just make sure the 3D you add actually helps your audience see what you’re trying to show instead of getting in the way.

How We Compare Differences

So what have we learned so far? That our brains are pretty good at seeing differences in color and contrast, but less good at interpreting differences in shape and perspective. Generally speaking, it takes us longer to understand a data visualization when there are more differences to process. So help a brain out and minimize the differences! One way to do this is to be consistent in your graphs.

Consistent Alignment

Our brains are fairly good at comparing differences in length, but only when things share a common reference point.Cleveland, William S. and Robert McGill. "Graphical Perception and Graphical Methods for Analyzing Scientific Data." Science 229.4716 (1985): 828-833. Web. 16 June 2014. For example, try to compare how long the bars are in the following graph:

Now try to do the same thing for this graph:

Wasn’t it easier to compare the bars in the second graph? Even though both bar graphs have a horizontal axis with labels and tick marks, the second graph minimizes the differences our brains have to process by making all of the bars start from the same reference point. Consistent alignment allows our brains to focus on the lengths without having to worry about different starting points for each bar.

Consistent Units

There may come a time when you want to show two sets of data in the same graph: something like a double line graph or a double bar graph, for instance. This is fine as long as you stick to the principle of consistency.

For example, the following bar graph compares how much it would cost to buy a few magical things in the United States versus the United Kingdom:

It seems like things are a bargain in the UK! But hold on because in this graph, it’s not enough to simply compare the bar lengths. There’s another difference we have to account for: the different units on the vertical axes. On the left side, the units are in US dollars. On the right side, the units are in euros. So in order to accurately compare prices, we need to either convert dollars to euros or euros to dollars. Some people can do this kind of currency conversion on the spot, and they are to be commended for their admirable math skills. The rest of us are going to look at that task and wonder why the maker of this bar graph is being so cruel to us. So don’t be cruel. Convert different units to consistent units so that your audience doesn’t have to. Like this:

As it turns out, magical things cost the same in the US and the UK.

Consistent Scales

In addition to having consistent units, your graphs should have consistent scales. For example, the bar graph below compares how long it takes an Earthling and a Martian to master different superpowers:

The heights of the bars make it seem like Martians take longer to master superpowers, but again, the vertical axes are different. Even though the units are consistent this time (both axes use hours), the scales are different. On the left side, the scale is in thousands of hours for Earthlings. On the right side, the scale is in hundreds of hours for Martians. The same data with consistent scales would look like this:

A graph with consistent scales makes it visually obvious that Martians master superpowers much more quickly than Earthlings do. Maybe it’s because they have less gravity on their planet.

Transparency

When it comes to data visualization and transparency, keep in mind that the data we show are just as important as the data we don’t. We want to give readers enough context and any other additional information they may need to accurately interpret our graphs.

Cumulative Graphs

A cumulative value tells you how much so far: it’s a total number that adds up as you go along. For example, let’s say you’re keeping track of how many superhero sightings there are in your neighborhood:

Month

Number of Superhero Sightings

Cumulative Number of Superhero Sightings

January
60
60

February
40
100 = 60 + 40

March
38
138 = 60 + 40 + 38

April
30
168 = 60 + 40 + 38 + 30

May
25
193 = 60 + 40 + 38 + 30 + 25

June
26
219 = 60 + 40 + 38 + 30 + 25 + 26

July
20
239 = 60 + 40 + 38 + 30 + 25 + 26 + 20

August
18
257 = 60 + 40 + 38 + 30 + 25 + 26 + 20 + 18

September
30
287 = 60 + 40 + 38 + 30 + 25 + 26 + 20 + 18 + 30

October
62
349 = 60 + 40 + 38 + 30 + 25 + 26 + 20 + 18 + 30 + 62

November
75
424 = 60 + 40 + 38 + 30 + 25 + 26 + 20 + 18 + 30 + 62 + 75

December
90
514 = 60 + 40 + 38 + 30 + 25 + 26 + 20 + 18 + 30 + 62 + 75 + 90

The first column of values gives you the number of sightings in each month, independent of the other months. The second column gives you the number of sightings so far by adding the numbers as you go down the column. For example, in January there were 60 superhero sightings, so the total so far is 60. In February there were 40 sightings, which means the total so far is 100 because the 40 sightings in February get added to the 60 sightings in January. And so on and so forth for each month that follows.

Why are we doing all this math? Because there’s something called a cumulative graph, which shows cumulative values.

If you’re going to show people a cumulative graph, it’s important that you tell them it’s a cumulative graph.

Why? Because this is what a regular line graph of the monthly, non-cumulative values looks like:

Notice how the line dips and rises: the number of monthly sightings generally decreases through August, then increases until the end of the year. Superheroes are apparently busier during the holiday months.

Compare the regular line graph above with the cumulative graph below:

Do you see how the general decrease in superhero activity from January through August is not obvious in the cumulative graph? The line in the cumulative graph keeps going up because the numbers keep adding up and getting bigger. This upward trend masks other patterns in the data. It’s easy to look at a cumulative graph and mistakenly think it’s showing a trend that consistently increases when that’s not really the case.

So when it comes to cumulative graphs, if you show it, let them know it, or else people might think the numbers are going up, up, and away.

Context

Without context, the stories our graphs tell have no point of reference that we can use to make a comparison. For example, is the following trendline average or unusual?

Alternatively, too much data can overwhelm an audience. In this case, the context gets lost among all the information and instead becomes an overcomplicated truth. For example, isn’t it harder to pick out the red trendline in the graph below?

This is why data visualization is challenging! You have to figure out how to present your data in a way that allows readers to clearly see trends and accurately compare differences. For example, let’s take a look at the following graph:

If you want to show your neighbors how the number of superhero sightings has changed over time, then presenting data that runs the course of a year instead of a few months provides more context. And instead of showing trendlines for dozens of cities around the world, your graph will be less cluttered and more relevant to your audience if you just show the trendline for your neighborhood. Finally, it’s a good idea to include some point of reference—like a trendline for the global average—so that your neighbors get a sense of how typical or atypical their neighborhood is.

The next and last chapter of this book goes into further detail about the importance of context and points out other ways graphics can mislead an audience. Just remember: being consistent and transparent goes a long way in your visualizations.

</div>

Common Visualization Mistakes

Chapter 18

Common Visualization Mistakes

By Kathy Chang, Kate Eyler-Werve, and Alberto Cairo

Welcome to the last (but certainly not least) chapter of the book! We hope you’ve learned enough to appreciate how much good data and good design can help you communicate your message. Professional data visualizers get excited by the stories they want to tell as well, but sometimes they forget to follow some best practices while doing so. It happens to the best of us. So in this chapter, we’re going to cover what those best practices are.

Don’t Truncate Axes

One of the ways a graph can be distorted is by truncating an axis. This happens when an axis is shortened because one or both of its ends gets cut off.

Sometimes a distortion like this is really obvious. For example, let’s say there are two allergy medicines called Happajoy and Pollaway. The bar graph below compares how effective these two medicines are at reducing the tearful, congested misery known as allergy symptoms. If you quickly glance at the bars, you may think that Happajoy is twice as effective as Pollaway is because its bar is twice as tall. But if you examine the graph more closely, you’ll see that the y-axis is truncated, starting from 30.2 and going up to only 30.7 percent. The truncated y-axis makes the difference between the two bars look artificially high. In reality, Happajoy’s effectiveness is only 0.2% higher than Pollaway’s, which is not as impressive as the results implied by the bar graph.

Sometimes a truncated axis and the resulting distortion can be more subtle. For example, the next graph shows the quantity of Happajoy sold from January through April 2014.

At first glance, there doesn’t appear to be a truncation issue here. The y-axis starts at zero, so that’s not a problem. The critical thing to understand is that it’s the x-axis that’s been truncated this time: we’re seeing sales from less than half the year. Truncating a time period like this can give the wrong impression, especially for things that go through cycles. And—you guessed it—the sale of allergy medicine goes through a seasonal cycle since allergy symptoms are typically higher in the spring and lower in the winter.

What would be a better way to show sales of Happajoy? Something like the graph below:

This graph shows the same dataset of Happajoy sales, except this time the y-axis is proportional and the x-axis covers two full years instead of just a few months. We can clearly see that sales of Happajoy went down in the winter and up in the spring, but that the rate of sales didn’t change much from year to year. In fact, sales were a little lower in 2014 than in 2013.

When you compare the last two graphs of Happajoy sales, do you see how different their stories are? If you were an investor in Happajoy’s company and you saw the graph with truncated axes, you might dance happily through the streets because it seems like the company is doing really well. On the other hand, if you saw the graph with proportional axes, you might reach for some aspirin instead of Happajoy because of the headache you’d get worrying about the overall decrease in sales.

So watch out for truncated axes. Sometimes these distortions are done on purpose to mislead readers, but other times they’re the unintentional consequence of not knowing how truncated axes can skew data. 

Don’t Omit Key Variables

The first principle is that you must not fool yourself — and you are the easiest person to fool.

 - Richard Feynman, 1974 Caltech Graduation Address

You know how famous people are sometimes criticized for something they said, and they often reply that they were quoted out of context? Context is important, especially when it comes to data. It’s very easy to fool yourself by leaving out variables that could affect how you interpret the data. So whenever you’re examining a variable and its relationships, carefully consider the ecosystem in which that variable exists and deliberately seek out other variables that could affect the one you’re studying.

This is easier said than done. For example, the map below shows each state’s market leader in allergy medicine: Happajoy is the leader in dark blue states, while Pollaway is the leader in light blue states. On the surface, it might seem like Happajoy is the market leader nationally, ahead of Pollaway. But to get the complete picture you have to pay attention to other variables.

For example, the bar graph below shows a breakdown of market share in each state. (We’re only going to look at the western continental states for now.)  The margins by which Happajoy leads are significantly less than the margins by which Pollaway leads.

Combine the information from the bar graph with the table below. The total sales in states where Happajoy is the leader is also significantly less than the total sales in states where Pollaway is the leader. When you add up the numbers, Pollaway’s total sales are more than twice that of Happajoy’s. Assuming that a similar pattern holds for the rest of the country, would it still be accurate to say that Happajoy is the national market leader in allergy medicine?

States | Happajoy | Pollaway
Wyoming (WY) | 299,734 | 219,037
North Dakota (ND) | 349,814 | 279,851
South Dakota (SD) | 408,343 | 341,675
Montana (MT) | 482,400 | 422,100
Idaho (ID) | 782,040 | 654,360
Nebraska (NE) | 872,320 | 798,080
New Mexico (NM) | 1,043,000 | 834,400
Nevada (NV) | 1,489,860 | 993,240
Utah (UT) | 1,313,300 | 1,256,200
Kansas (KS) | 1,414,140 | 1,183,260
Oklahoma (OK) | 1,907,500 | 1,526,000
Oregon (OR) | 2,027,480 | 1,481,620
Arizona (AZ) | 3,014,380 | 2,883,320
Colorado (CO) | 1,297,000 | 3,372,200
Washington (WA) | 2,069,100 | 4,138,200
Texas (TX) | 5,733,200 | 17,720,800
California (CA) | 7,608,000 | 26,628,000
Sales Totals | 32,111,611 | 64,732,343

The lesson here is that if you want to provide a fair picture of what’s going on, you have to expand your scope to show the variables that put things in their proper context. That way, you can provide your readers with a more complete and nuanced picture of the story you’re trying to tell.

Don’t Oversimplify

Life is complicated, right? Data can be complicated, too. Complicated is hard to communicate, so it’s only natural to want to simplify what your data are saying. But there is such a thing as simplifying too much. Oversimplifying is related to the previous point about not expanding the scope enough to provide a clear picture of what’s going on.

For example, let’s say you’re an investor of RediMedico, the maker of Happajoy, and you attend the annual sales meeting. The CEO of RediMedico starts off the presentation with the following graphic:

Now the investor in you might look at that graphic and start daydreaming about all the wonderful things you’ll do with such a great return on your investment. But then the data pro in you kicks in, and you start wondering about what that 18% increase means. You ask yourself:

Compared to what?

Compared to when?

Compared to whom?

These are all worthwhile questions to answer with a visualization! Thankfully, the CEO of RediMedico agrees and presents the next graphic, which compares the revenues from the five top-selling medicines RediMedico makes:

If we do some number-crunching, we see that the average increase in revenue between 2013 and 2014 is indeed 18%. However, we also see that this increase is primarily due to a whopping 225% increase in revenue from a single medicine, Exoalgio. Revenue from 3 out of 5 medicines actually dropped. So RediMedico’s first graphic tells part of the truth, while the second graphic tells the whole truth by presenting the details behind the single number. 

Using a graphic with a single number and no breakdowns is like writing a news headline without the news story. Keep the headline—that revenue improved by 18%—and then provide the context and the background to flesh out the full story. Try to be true to the underlying complexity by digging deeper into the data and providing readers with a better understanding of the numbers you’re presenting.

Don’t Choose the Wrong Form

Creating a data visualization is a balancing act between form and function. When choosing a graphic format for your data, you’ll have to figure out how to effectively communicate to your audience in an aesthetically pleasing way. This may seem like a daunting task, but fear not! There’s actually a lot of research that can help us with this. In fact, you’ve already been introduced to some of this research in the previous chapter: Cleveland and McGill’s “Graphical Perception” paper, in which they rank how well different graphic forms help people make accurate estimates. 

You can use this scale to help you choose the best graphic representation of your data. Area and shading are good at giving readers a general overview that helps them understand the big picture. Graphs with a common baseline (such as bar graphs) are good at helping readers make accurate comparisons.

Since we’ve already looked at examples of bar graphs and line graphs in this chapter, let’s take a look at a couple graphics that use area and shading.

The bubble graphic uses area to display the units sold of the top selling allergy medicine in some states. Based on bubble size, you can generally tell that more Happajoy was sold in Arizona than in New Mexico. But can you tell by how much? Is the Arizona bubble three times bigger than the New Mexico bubble? Four times? It’s hard to tell. It’s even harder to tell when the bubble sizes are closer together: Did Utah or Kansas sell more Happajoy?

We run into the same problem with the next graphic, which uses shading to represent Happajoy sales: California is definitely darker than Texas, but how much darker? Two times? Three times? Who knows? This is why area and shading are better for giving an overall picture instead of making precise comparisons.

In addition to area and shading, angles also give us a tough time when it comes to making accurate estimates. This is why it’s so hard to compare two pie charts, as in the example below.

It’s already hard to precisely compare the slices within the same pie chart. It’s even harder to compare slices across different pie charts. If the goal of this graphic is to help readers compare revenues from one year to the next, then something like a bar chart would have been a better choice.

That’s the key thing: think about which graphic forms will best facilitate the tasks you want your readers to do.

Do Present Data in Multiple Ways

We just covered how different graphic forms are good at doing different things. So what do you do when you have a lot of data and you want to show different aspects of those data, but you also don’t want to overwhelm your audience with an overly complicated graphic? One way to deal with this challenge is to present your data in multiple ways. You can individually show multiple graphics, each one showing a different aspect of the data, so that taken together your audience gets a more accurate picture of the data as a whole.

For example, let’s say the CEO of RediMedico wants to show investors how Happajoy has been selling in the United States since it was first introduced to the market ten years ago. The available data consists of Happajoy sales in every state for every year since 2004. You’re the lucky data pro who gets to figure out how to present both the big picture and the small picture inside this data.

Let’s start with the big picture. Remember how graphic forms that use area or shading are best at giving a general overview? For every year that Happajoy has been on the market, a map that uses shading to represent sales can give people a general sense of how sales have changed across time and location:

Now let’s move on the small picture. Let’s say RediMedico started to advertise heavily in California and New York a few years ago, and the investors are wondering how sales in those states are doing. Using the same dataset, you can give a more detailed picture of the sales in one state:

Or you can compare the sales between different states:

See? Same data, different presentations. So if you want to show your readers different sides of the same story, give them multiple graphic forms.

Do Annotate with Text

They say that a picture is worth a thousand words, but that doesn’t mean you should forget about words entirely! Even your most beautiful and elegant visualizations can use text to highlight or explain things to your audience. This is especially useful when you’re presenting multiple layers of information because your annotations can help readers connect the various pieces into an understandable whole. And obviously, you’re not reading this book in order to make boring visualizations, right? Good visualizations engage an audience, so adding text is a great way to address questions that may come up as curious readers examine your graphic.

For example, let’s go back Happajoy sales. If you see a graphic like the following:

Then you might be wondering what happened between 2009 and 2010. Why was there such a sharp drop in revenue? In this case, it would be helpful to add some text:

So whenever you create a visualization, think about the “So what?” of your graphic: Why should people care about the information you’re presenting? Add annotations to help them understand why they should care. Write a good headline that catches their attention, a good introduction that highlights interesting data points, and a good narrative that structures your visualization logically. Good writing is an important part of good visualizations.

Case Study of an Awesome Infographic

To close out this chapter, let’s take a look at all of these pro tips in action by going through a visualization made by real data pros: an infographic by The New York Times about breast cancer. The designers organized the information as a narrative with a step-by-step structure. This is an interactive graphic, so it’s best if you click through the link to get the full experience.

On the first screen, you see a bubble graphic that gives you a general sense of which countries have the most new cases of breast cancer . After clicking “Begin”, you see a scatterplot with proportional axes. The scatterplot shows that there is an inverse correlation between breast cancer detection and mortality: as more women are detected with breast cancer, fewer women die from it. A scatterplot is a good way to show correlations between variables, and a bubble graphic is a good way to show a general overview, so the designers chose graphic forms that matched well with what they wanted to show.

Notice how the designers use text to write a good headline that grabs the reader’s attention (“Where Does Breast Cancer Kill?”) and to highlight another aspect of this scatterplot—that highly developed countries have higher diagnosis rates (and lower mortality rates) while the opposite is true for the developing world. As you keep clicking “Next”, the designers guide you deeper into the scatterplot by highlighting a cluster of countries and providing an annotation that gives you further insight into that cluster. Without these notes, we would be left with a relatively straightforward scatterplot that leaves little room for us to explore the data further. By providing useful and well-placed annotations, the designers help us see relationships that we otherwise may have missed.

The designers also present the data in multiple ways. They use color to add another layer of detail: the development status of various countries. In addition, if you’re curious about the statistics for a specific country, you can mouse over that country’s dot to get those numbers.

Finally, by adding useful annotations and showing the data in multiple ways, the designers present the data within a context that doesn’t leave out important variables or oversimplify. By following through on some good data visualization practices, the designers created a clear, balanced, and engaging infographic.

</div>

Resources

Resources

Chapter 3: Intro to Survey Design

Online survey sites

Kwik Surveys, http://www.kwiksurveys.com

Free Online Surveys, http://www.freeonlinesurveys.com

Google Forms, http://www.google.com/google-d-s/createforms.html

Survey Gizmo, http://www.surveygizmo.com

Survey Monkey, https://www.surveymonkey.com

Survey Planet, https://www.surveyplanet.com

Chapter 4: Types of Survey Questions

Conducting Surveys

Dierckx, Didier. "The Case of 'Likert Scales v. Slider Scales,' Market Research (blog) Checkmarket.com, https://www.checkmarket.com/2012/08/likert_v_sliderscales/

Fink, Arlene. How to Conduct Surveys: A Step-by-Step Guide, 3rd ed., Thousand Oaks, California: SAGE Publications, Inc., 2006.

Fowler, Floyd J. Survey Research Methods, 5th ed., Thousand Oaks, California: SAGE Publications, Inc., 2014.

Gray, George and Neil Guppy. Successful Surveys: Research Methods and Practice, 3rd ed., Scarborough, Ontario: Thomson Nelson, 2003.

"Scaled Questions," Instructional Assessment Resources, The University of Texas at Austin, http://www.utexas.edu/academic/ctl/assessment/iar/teaching/plan/method/survey/responseScale.pdf

Chapter 7: Getting Data Ready for Cleaning

Splitting delimited text into columns in Excel

"Split text into different cells," Microsoft Office Support, http://office.microsoft.com/en-us/excel-help/split-text-into-different-cells-HA102809804.aspx

Splitting strings

JavaScript

"JavaScript String split() Method," W3Schools, http://www.w3schools.com/jsref/jsref_split.asp

Visual Basic

"Split Function (Visual Basic)," Microsoft Developer Network, http://msdn.microsoft.com/en-us/library/6x627e5f(v=vs.90).aspx

String Operations in Python

"Common string operations," The Python Standard Library, Python Software Foundation, https://docs.python.org/2/library/string.html

Missing Data

Osborne, Jason W., "Dealing with Missing or Incomplete Data: Debunking the Myth of Emptiness," Ch 6, Best Practices in Data Cleaning, Los Angeles, CA: SAGE Publications, Inc, 2013:

Chapter 8: Data Cleaning

Spreadsheet software

"List of spreadsheet software," Wikipedia, http://en.wikipedia.org/wiki/List_of_spreadsheet_software

Regular Expressions

Guides and Testers

"Regular Expressions - User Guide," Zytrax, http://www.zytrax.com/tech/web/regex.htm

RegExr v2.0, http://www.regexr.com

JavaScript

"JavaScript RegExp Reference," W3Schools, http://www.w3schools.com/jsref/jsref_obj_regexp.asp

Python

"Regular expression operations," The Python Standard Library, Python Software Foundation, https://docs.python.org/2/library/re.html

Chapter 10: What Data Cleaning Can and Can't Catch

Data Transformations

"Transformations to Achieve Linearity," Stat Trek, http://stattrek.com/regression/linear-transformation.aspx#

Glossary

詞彙表

聚集偏差 Aggregation bias

聚集偏差發生於不正確的推論從已聚集或匯總的資料導出時。包括基於整體本身的特性對整體的各部分特性作出推論。

聚集 Aggregation

聚集是指資料被蒐集並以某種方式匯總或按類別分類的過程。

應用統計 Applied statistics

應用統計是一種將統計方法應用於各學科和研究領域的統計類型。

算術平均值 Arithmetic mean

算術平均值，通常簡稱為平均值，是一種平均或集中趨勢的測量，其資料集中間決定於加總所有數值後，再除以資料的個數。

態度陳述 Attitudinal statement

態度陳述是在一個調查的量表問題上，要求一個人來評價他或她對一個特定主題的感受。

軸標籤 Axis label

An axis label is used on a graph to denote the kind of unit or rate of measurement used as the dependent or independent variable (or variables), and can be found along an axis of a graph. 軸標籤用於圖表上來表示一種單位或作為應變數或自變數（或變數群）的測量率，可以沿著圖形的軸線找到。

反向轉換 Back transformation

反向轉換是以數學運算應用於資料中那些已經被轉換的資料集，為了反向轉換或回復該資料到其原始形式的過程。

後端檢查 Back-end check

後端檢查也稱為伺服器端檢查，是一種通過電子手段蒐集之資料集的資料驗證，在後端執行，或在資料存儲在電子資料庫之後。

長條圖 Bar graph

長條圖或圖表使用水平或垂直條形，以其長度比例性地表示資料集中的值。具有垂直條形的圖表也稱為柱狀圖或圖表。

統計地圖 Cartogram

統計地圖是以類別資料覆蓋投影，並使用不同的顏色來表示各個類別的地圖。不同於熱圖，統計地圖不一定使用色彩飽和度來描繪一個類別中值的頻率。

類別資料 Categorical data

類別資料是可被分類到相異類別的計量或屬性資料。

類別標籤 Category label

類別標籤用於圖表來表示資料類別或群組的名稱，可以是描述性的或一個數值範圍。

圖表標題 Chart title

圖表標題是賦予圖表的描述，包括針對目標對象的訊息摘要，也可以包括資料集相關資訊。A chart title is the description assigned to the graph and includes a summary of the message aimed at the target audience and may include information about the dataset.

選取方塊回應 Checkbox response

A checkbox response refers to an answer given to a question in a survey administered in electronic form, for which one or more responses can be selected at a time, as may be indicated by a checkbox an individual clicks on.

彩度 Chroma

Chroma is the saturation, or vividness, of a hue.

封閉式問題 Closed question

A closed, or closed-ended, question is a type of question featured in a poll or survey that requires a limited, or specific kind of, response and is used to collect quantitative data or data that can be analyzed quantitatively later on.

編碼簿 Codebook

A codebook documents the descriptions, terms, variables, and values that are represented by abbreviated or coded words or symbols used in a dataset, and serves as a means for coding and decoding the information.

色彩理論 Color theory

Color theory refers to principles of design focuses on colors and the relationships between them.

連續變數 Continuous variable

A continuous variable, or continuous scale, has an unlimited number of possible values between the highest and lowest values in a dataset.

相關 Correlation

Correlation measures the degree of association, or the strength of the relationship, between two variables using mathematical operations.

CRAAP 測試

The CRAAP test denotes a set of questions a researcher may use to assess the quality of source information across five criteria: currency, relevance, authority, accuracy, and purpose.

資料清理 Data cleaning

Data cleaning, also called data checking or data validation, is the process by which missing, erroneous, or invalid data are determined and cleaned, or removed, from a dataset and follows the data preparation process.

資料標籤 Data label

A data label is used on a graph to denote the value of a plotted point.

資料準備 Data preparation

Data preparation is the process by which data are readied for analysis and includes the formatting, or normalizing, of values in a dataset.

資料轉換 Data transformation

Data transformation is the process by which data in a dataset are transformed, or changed, during data cleaning and involves the use of mathematical operations in order to reveal features of the data that are not observable in their original form.

資料視覺化 Data visualization

Data visualization, or data presentation, is the process by which data are visualized, or presented, after the data cleaning process, and involves making choices about which data will be visualized, how data will be visualized, and what message will be shared with the target audience of the visualization. The end result may be referred to as a data visualization.

資料 Data

Data are observations, facts, or numeric values that can be described or measured, interpreted or analyzed.

應變數 Dependent variable

A dependent variable is a type of variable whose value is determined by, or depends on, another variable.

敘述統計 Descriptive statistics

Descriptive statistics is a type of applied statistics that numerically summarizes or describes data that have already been collected and is limited to the dataset.

日誌 Diary

A diary is a data collection method in which data, qualitative or quantitative, are tracked over an extended period of time.

二分問題 Dichotomous question

A dichotomous question is a type of closed question featured in a poll or survey that requires an individual to choose only one of two possible responses.

直接測量 Direct measurement

Direct measurement is a type of measurement method that involves taking an exact measurement of a variable and recording that numeric value in a dataset.

離散變數 Discrete variable

A discrete variable, or a discrete scale, has a limited number of possible values between the highest and lowest values in a dataset.

外部資料 External data

External data refer to data that a researcher or organization use, but which have been collected by an outside researcher or organization.

類事實 Factoid

A factoid, or trivial fact, is a single piece of information that emphasizes a particular point of view, idea, or detail. A factoid does not allow for any further statistical analysis.

篩選器 Filter

A filter is a programmed list of conditions that filters, or checks, items that meet those conditions and may specify further instructions either for the filtered items.

焦點團體 Focus group

A focus group is a data collection method used for qualitative research in which a group of selected individuals participate in a guided discussion.

必填問題 Forced question

A forced question is a type of scaled question featured in a survey that requires an individual to choose from a give range of possible responses, none of which is neutral.

前端檢查 Front-end check

A front-end check, also called a client-side check, is a type of data validation for datasets gathered electronically, and is performed at the front end, or before data are stored in an electronic database.

圖形使用者介面 Graphical user interface (GUI)

A graphical user interface, or GUI, is a type of interface that allows a user to interact with a computer through graphics, such as icons and menus, in place of lines of text.

熱圖 Heat map

A heat map is a graph that uses colors to represent categorical data in which the saturation of the color reflects the category’s frequency in the dataset.

直方圖 Histogram

A histogram is a graph that uses bars to represent proportionally a continuous variable according to how frequently the values occur within a dataset.

色相 Hue

A hue, as defined in color theory, is a color without any black or white pigments added to it.

自變數 Independent variable

An independent variable is a type of variable that can be changed, or manipulated, and determines the value of at least one other variable.

推理統計 Inferential statistics

Inferential statistics is a type of applied statistics that makes inferences, or predictions, beyond the dataset.

資訊圖表 Infographic

An infographic is a graphical representation of data that may combine several different types of graphs and icons in order to convey a specific message to a target audience.

互動圖像 Interactive graphic

An interactive graphic is a type of visualization designed for digital or print media that presents information that allows, and may require, input from the viewer.

訪談者效應 Interviewer effect

Interviewer effect refers to any effect an interviewer can have on subjects such that he or she influences the responses to the questions.

無效資料 Invalid data

An invalid data are values in a dataset that fall outside the range of valid, or acceptable, values during data cleaning.

引導性問題 Leading question

A leading question is a type of question featured in a poll or survey that prompts, or leads, an individual to choose a particular response and produces a skewed, or biased, dataset.

圖例 Legend

A legend is used on a graph in order to denote the meaning of colors, abbreviations, or symbols used to represent data in dataset.

可辨識性 Legibility

Legibility is a term used in typography and refers to the ease with which individual characters in a text can be distinguished from one another when read.

線圖 Line graph

A line graph uses plotted points that are connected by a line to represent values of a dataset with one or more dependent variables and one independent variable.

中位數 Median

A median is a type of average, or measure of central tendency, in which the middle of a dataset is determined by arranging its numeric values in order.

元資料 Metadata

Metadata are data about other data, and may be used to clarify or give more information about some part or parts of another dataset.

遺漏資料 Missing data

Missing data are values in a dataset that have not been stored sufficiently, whether blank or partial, and may be marked by the individual working with the dataset.

衆數 Mode

A mode is a numeric value that appears most often in a dataset.

動態圖像 Motion graphic

A motion graphic is a type of visualization designed for digital media that presents moving information without need for input from the viewer.

多系列 Multiseries

A multiseries is a dataset that compares multiple series,or two or more dependent variables and one independent variable.

常態分布 Normal distribution

A normal distribution, often called a bell curve, is a type of data distribution in which the values in a dataset are distributed symmetrically around the mean value. Normally distributed data take the shape of a bell when represented on a graph,the height of which is determined by the mean of the sample, and the width of which is determined by the standard deviation of the sample.

開放內容 Open content

Open content, open access, open source, and open data are closely-related terms that refer to digital works that are free of most copyright restrictions. Generally, the original creator has licensed a work for use by others at no cost so long as some conditions, such as author attribution, are met (See: Suber, Peter. Open Access, Cambridge, Massachusetts: MIT Press, 2012). Conditions vary from license to license and determine how open the content is.

開放性問題 Open question

An open, or open-ended question, is a type of question featured in a survey that does not require a specific kind of response and is used to collect qualitative data.

顺序偏差 Order bias

Order bias occurs when the sequencing of questions featured in a survey has an effect on the responses an individual chooses, and produces a biased, or skewed, dataset.

異常值 Outlier

An outlier is an extremely high or extremely low numeric value that lies outside the distribution of most of the values in a dataset.

樣式比對 Pattern matching

Pattern matching is the process by which a sequence of characters is checked against a pattern in order to determine whether the characters are a match.

點對點網路 Peer-to-peer (P2P) network

A peer-to-peer network, often abbreviated P2P, is a network of computers that allows for peer-to-peer sharing, or shared access to files stored on the computers in the network rather than on a central server.

圓餅圖 Pie chart

A pie chart is a circular graph divided into sectors, each with an area relative to whole circle, and is used to represent the frequency of values in a dataset.

群體 Population

A population is the complete set from which a sample is drawn.

或然率 Probability

Probability is the measure of how likely, or probable, it is that an event will occur.

質性資料 Qualitative data

Qualitative data are a type of data that describe the qualities or attributes of something using words or other non-numeric symbols.

計量資料 Quantitative data

Quantitative data are a type of data that quantify or measure something using numeric values.

選取回應 Radio response

A radio response refers to an answer given to a question in a poll or survey administered in electronic form, for which only one response can be selected at a time, as may be indicated by a round radio button an individual clicks on.

範圍檢查 Range check

A range check is a type of check used in data cleaning that determines whether any values in a dataset fall outside a particular range.

範圍 Range

A range is determined by taking the difference between the highest and lowest numeric values in a dataset.

原始資料 Raw data

Raw data refer to data that have only been collected, not manipulated or analyzed, from a source.

可讀性 Readability

Readability is a term used in typography and refers to the ease with which a sequence of characters in a text can be read. Factors affecting readability include the placement of text on a page and the spacing between characters, words, and lines of text.

抽樣 Sample

A sample is a set of collected data.

抽樣偏差 Sampling bias

Sampling bias occurs when some members of a population are more or less likely than other members to be represented in a sample of that population.

量表問題 Scaled question

A scaled question is a type of question featured in a survey that requires an individual to choose from a given range of possible responses.

散佈圖 Scatterplot

A scatterplot uses plotted points (that are not connected by a line) to represent values of a dataset with one or more dependent variables and one independent variable.

序列圖 Series graph

A series graph proportionally represents values of a dataset with two or more dependent variables and one independent variable.

序列 Series

A series is a dataset that compares one or more dependent variables with one independent variable.

加深 Shade

Shade refers to adding black to a hue in order to darken it.

偏斜資料 Skewed data

Skewed data are data with a non-normal distribution and tend to have more values to the left, as in left-skewed, or right, as in right-skewed, of the mean value when represented on a graph.

堆疊長條圖 Stacked bar graph

A stacked bar graph is a type of bar graph whose bars are divided into sub-sections, each of which proportionally represent categories of data in a dataset that can be stacked together to form a larger category.

標準差 Standard deviation

A standard deviation is a measure of how much the values in a dataset vary, or deviate, from the arithmetic mean by taking the square root of the variance.

靜態圖像 Static graphic

A static graphic is a type of visualization designed for digital or print media that presents information without need for input from the viewer.

統計學 Statistics

Statistics is the study of collecting, measuring, and analyzing quantitative data using mathematical operations.

可和多系列 Summable multiseries

A summable multiseries is a type of multiseries with two or more dependent variables that can be added together and compared with an independent variable.

摘要記錄 Summary record

A summary record is a record in a database that has been sorted,or aggregated, in some way after having been collected.

淡化 Tint

Tint refers to adding white to a hue in order to lighten it.

交易記錄 Transactional record

A transactional record is a record in a database that has not yet been sorted, or aggregated, after collection.

明度（色彩） Value (color)

Value, or brightness, refers to the tint, shade, or tone of a hue that results black or white pigments to a base color.

變量 Variance

Variance, or statistical variance, is a measure of how spread out the numeric values in a dataset are, or how much the values vary, from the arithmetic mean.

Contributors

Contributors

Mihai Badescu

Web Developer

research, code

Mihai is a junior web developer living in Chicagoland. If he’s not dreaming of working with data, he’s off somewhere actually working with it. He also has a thing for 90s Japanese cooking shows.

@mihaibad

Vitor Baptista

Data Visualization Developer

Open Knowledge Foundation

editor

Vitor Baptista is a data visualization developer at the Open Knowledge Foundation working on the CKAN data portal platform. He sees the access, consumption, and interpretation of data of all kinds as an exciting way to help people understand the world and make it better.

@vitorbaptista website

Petr Beles

Senior ETL Consultant

2150 GmbH

editor

Petr is an expert in Data Modeling, ETL Architecture and Development. He’s currently focused on agile business intelligence, including automated testing and data vault modeling. Always eager to share and expand his knowledge, Petr is a regular speaker at TDWI conferences in Switzerland and Germany.

website

Jessica Berger

Research Study Coordinator

Northwestern University Feinberg School of Medicine

technical editor

Jessica works in psychiatric clinical research and has a passion for building databases and queries. She also enjoys opportunities to write and edit presentations and publications on the subject. She recently began work toward a degree in epidemiology and biostatistics at Northwestern University Feinberg School of Medicine.

@nerdfighter_14

Marit Brademann

Data Enthusiast

writer

Marit’s interest in data is fueled by her support for transparency and open access to information. She holds a Master’s degree in quantitative sociology and regularly contributes to projects in the cultural, educational, and scientific spheres. She loves traveling, connecting, and writing about self-improvement, society, and, of course, (open) data.

Amanda Buenz Makulec

Monitoring & Evaluation Associate

John Snow Inc.

writer

Amanda works at the intersection of global health monitoring, evaluation, and communications. She loves how well-designed visualizations can make data accessible and even help improve health programs. Co-founder of the datavizhub.co community, she enjoys connecting with other viz enthusiasts to share ideas.

@abmakulec website

Alberto Cairo

Professor & Director of Visualization Program

University of Miami

writer

Alberto teaches infographics and visualization at the University of Miami School of Communication. He also serves as director of the visualization program in the University’s Center for Computational Science. Author of The Functional Art, Albert has led workshops and consulted media and educational institutions in more than 20 countries.

@albertocairo website

Michael A. Castello

PhD/MD Candidate

Loma Linda University

writer

Currently working toward a PhD in human physiology, Michael Castello researches Alzheimer’s disease and creates downright fascinating scientific presentations on the subject. Outside the lab, Michael writes, builds things, plays computer games, and hypothesizes about the future.

@castello website

Michelle Cazares

distribution

Michelle is currently dabbling in the business world, so learning how to work with data has been an interesting journey. She is set to start her MBA in Fall 2014 and hopes to be able to use the e-book to help her throughout her courses. In her spare time, she likes to read, play videogames (N7 at heart!), and learn about the world.

@andromeda_N7

Kathy Chang

editor, writer, distribution

The daughter of immigrants, Kathy remembers when NYC subway trains were covered in graffiti and kind of misses that. Nowadays she rides her trusty Brompton wherever she can. She loves bacon, hates cilantro, and enjoys learning about learning.

website

Jeana Chesnik

E-Learning Developer, Massage Envy Franchising, LLC

editor

Jeana designs, develops, and implements online training programs to support employee development at the corporate and national level. She is currently working with a team at the Franchise Support Center to establish learning and development metrics that evaluate the effectiveness of learning outcomes and make changes as needed.

@Jeanacee

Trina Chiasson

Co-founder & CEO

Infoactive

organizer

Trina is the co-founder of Infoactive and a fellow at the Reynolds Journalism Institute where she works to make data visualization more accessible for journalists. She enjoys turning numbers into stories and making data more human.

@trinachi website

Howard Coale

NY Practice Lead & Creative Director, Sapient Global Markets

writer

Howard leads the experienced, talented, ambitious, and (mostly) good-humored New York team at Sapient Global Markets, which focuses on data visualization and user experience design.

website

Ellen Cooper

Senior Researcher

Quadra Research

writer

Ellen has extensive experience designing, implementing, and managing market research studies. Twenty years ago, she launched Quadra Research to provide qualitative and quantitative research studies for industries including utilities, non-profits, packaged goods, real estate, and finance. She is a Certified Advertising Agency Practitioner, corporate member of the Marketing Research and Intelligence Association, and Certified Marketing Research Professional.

website

Bill Coppinger

Owner

WhaleSongServices

research

Bill Coppinger has over 30 years’ experience in Education and Training. He co-founded the I*EARN Network in Australia and chaired the International I*EARN Management Team for 3 years. He has a passion for supporting the uptake of evidence-based planning and data visualization. He owns and operates WhaleSongServices.

@billcoppinger website

Colleen Cressman

editor

A graduate student in Library & Information Science at Simmons College in Boston, MA, Colleen is interested in the effects of open access on resources for education, research and scholarship. She also studies the relationship between digital literacy and learning.

@namsserc

Alistair Croll

Founder

Solve For Interesting

writer

Alistair works in web performance, big data, cloud computing, and startup acceleration. Co-founder of Networkshop and Coradiant, he’s helped create several startups, accelerators, and conferences. He chairs various tech events, including O’Reilly’s Strata conference and the International Startup Festival, and has written four books on analytics, technology, and entrepreneurship, including Lean Analytics.

@acroll website

Jared Crooks

President

NOURI

technical editor

Jared is a scientist and innovator working in emerging technology, energy, and big data. He is the co-founder of NOURI—an “out of this world” food company with the mission of making the world a better place by ending child hunger—and the author of several children’s books.

@jacrooks website

Ersin Demirok

Sabancı University

writer

Ersin lives a double life, well sort of. He’s a researcher in the management and organization department of a national university and also a business developer for investors in the mining and energy industry. He digs being able to present any sort of investment opportunity clearly and quickly with inspiring and simple data visualizations.

@demiroker

Mizel Djukic

Senior Product Manager

Millennial Media

writer

Mizel is a Senior Product Manager in the mobile tech industry. He builds tools that use big data to tell stories effectively and allow users to make better, data-driven business decisions.

@mizthediz

Candice Douglas

Chief of Social at The Social Post

editor

Candice’s love for putting words together beautifully began early in her life. Her love for photography came next. Today, she is paid to chat on social media, capture light through her camera lens, and help people translate their message into words for the rest of the world to read.

@TheSocialPostSA website

Steve Drew

Co-Founder, Veraison Vineyards

technical editor

Quite the globetrotter, Steve lived in New York and London before moving to South Africa in 2007 where he bought a Burgundian-sized vineyard and farm house five minutes from the Franschhoek village center.

Oz du Soleil

DataScopic

writer

Oz is a Chicago-based freelancer who has been working with Excel and data for 15 years. He’s a passionate speaker and trainer devoted to data literacy. When Oz isn’t elbow-deep in data, he’s known for sriracha, bowties, and good bourbon.

@Oz_Dragoncookie website

Kate Eyler-Werve

Project Manager, Mightybytes

editor, writer

Kate helps organizations build websites that both deliver against business objectives AND look good. Kate also trains people on methods and models for integrating content strategy, user experience, and web design. She’s co-authored two books: Return on Engagement: Content Strategy and Web Design Techniques for Digital Marketing, and O’Reilly Media’s The Civic Apps Competition Handbook.

@innokate

Jane Foo

Digital Systems Librarian, Seneca College

writer

With a background in user-centered design, Jane has experience in systems testing (including integration and usability), systems implementation, UI Design, and training. She credits an undergraduate course on environmental psychology with prompting a lifelong interest in human-computer interaction and the impact of technology and visual design on everyday activities.

@grumpel

Philipp Gassner

Interace Expert & Founder of Green Challenge Accepted

distribution

Philipp is an environmental scientist and communication expert with a passion for knowledge, people, and the environment. Dedicated to communicating the importance of sustainability, he turns data into drama, numbers into narrative, and stats into stories, translating knowledge into practice to change institutions.

@GrnChllngAccptd website

Collin Graves

Co-Founder, Vitalane

technical editor

Collin stumbled upon the art and psychology of design while working as a mechanic in the Air Force. He went on to found a few award-winning startups and now works as a startup consultant, UX designer, and freelance web developer.

@GravesCollin website

Dyanna Gregory

Co-Organizer, Data + Design

organizer, writer

Dyanna is a librarian and biostatistician who has been obsessive about data cleaning since the age of 13 and loves to teach statistics to people of all ages, especially those who think they hate math.

Tom Grubisich

Editorial Director, Local America

editor

Creator of and inspiration behind Local America, Tom is a veteran journalist with more than 30 years’ experience. His previous gigs include: editor at the Washington Post, managing editor of AOL local news, founder of a chain of community newspapers in metro Washington, and consulting Web editor at the World Bank.

@TomGrubisich website

Nidhal Haddad

Student, ESSAI

editor

Nidhal is pursuing a national degree at The Higher School of Statistics and Data Analysis in Tunisia (ESSAI). His current and past projects include PCA analysis, tree-based models, and network analysis. His thesis project focuses on market share prediction in the airline industry.

Margie Henry

Senior Analyst, Rakuten Marketing

writer

Margie’s relationship with data began in college when she nearly failed her first statistics class and decided the subject was a beast to be conquered. When she’s not cranking out reports and crunching numbers, Margie addresses the growing need for evidence-based decision making and performance measurement among organizations in the arts, culture, and commercial creative sectors.

Camilo Herrera

President, RADDAR

editor

Camilo serves as research consultant and strategic advisor to more than 100 brands. He has authored academic articles on microeconomics, cultural capital, and marketing, including several on the Columbian consumer.

website

Melissa Hill Dees

Programs & Communication, Volunteer Center of Morgan County

editor

Melissa is an integrator, collaborator, innovator, and relationship builder. She has 20 years of successful marketing experience working with industry leaders and 8 years in entrepreneurial electronic marketing and CRM for small businesses.

Saurabh Jain

Analytics Lead, Fractal Analytics

writer

Saurabh is an analytics lead for one of the largest consumer goods companies in the world. On a typical day, he tackles tough big data in order to improve decision management, predict and influence consumer behavior, and optimize business results.

@saurabhjain1099 website

Kate Kretschmann

Project Coordinator, Arizona Geological Survey

project manager

Kate coordinates community building and project development for the NSF EarthCube Initiative, a data and knowledge management system that will allow for cross-discipline data sharing across the geosciences. She comes to her work with a background in journalism, science reference publishing, and online media.

Ginette Law

writer

Ginette collects, explores, and reports data for companies and organizations, including the World Economic Forum, ictQatar, Huawei, and the World Wide Web Foundation. When not geeking out over data, Ginette enjoys the finer things in life… like ice cream.

@ggeeksd

An Le

Engagement Leader, Officience

editor

Born and living in South Central Vietnam in the time of Facebook and Twitter, Thanh An is an engagement leader at the global consulting firm Officience. He’s driven by a strong entrepreneurial spirit and a fascination with mass data digitization.

Bonnie Meisels

Designer, Social Entrepreneur, Real Estate Broker

other

As a graduate of Parson's School of Design, Bonnie designed handbags & accessories in New York. After moving back to Montreal, she is applying her passion for designing and making a difference to her practice as a socially conscious real estate broker.

Tom Morris

Product Manager & Software Engineer

technical editor

Tom is a product manager, software geek, and athlete who lives and works in Boston, Massachusetts.

@tfmorris

Callie Neylan

Senior Software Designer, Microsoft

writer

Callie was born in Aspen, Colorado, before movie stars discovered it. She’s worked at various startups, software companies, and design firms, and as a senior interaction designer at NPR. She now works with some of the best engineers in the world creating data visualization and visual analytics tools.

@neylano website

Sanjit Oberai

Deputy Editor, IndiaSpend

technical editor

Sanjit is Deputy Editor of IndiaSpend, a non-profit organisation and India’s first Data Journalism initiative which utilises open data to empower citizens decision making process. He likes creating stories around data with the help of free to use tools and believes that story telling is the way forward. He has worked with PaGaLGuY and Firstpost.com. He lives in Mumbai.

@sanjit_oberai website

Tara Phelps

Founder, Success Technologies

editor

Tara the Technology Matchmaker is a business automation and strategy expert whose passion is helping coaches, entrepreneurs, and small business owners use technology to save time and money, create more meaningful relationships with clients and prospects, and achieve greater success in business.

@taraphelps website

Kiran PV

writer

Kiran provides data science training at Jigsaw Academy, an Online School of Analytics based in Bangalore, India. He holds a dual degree in engineering from IIT Madras, India, and his technical expertise spans SAS, Python, Hadoop, and Tableau. Outside of work, Kiran writes, travels, and cheers for his favorite club, Manchester United in EPL.

Christine Ridgway

National Clinical Educator at Teleflex Medical Australia

editor

Christine’s passion is communicating complex topics in simple and approachable ways, empowering people to perform at their highest level. She is a National Clinical Educator at Teleflex Medical Australia, specializing in designing learning resources to support the sale and use of complex medical devices and, ultimately, improve patient outcomes.

Sarah Rigdon

Writer & Consultant

writer

Sarah likes translating complex ideas into good stories. A fan of civic tech and mass collaboration, she’s an alumna of NASA’s Open Innovation Program and a volunteer at Code for DC and Code for Houston. She led communications for the 2013 International Space Apps Challenge and helped organize the TechLady Hackathon + Training Day.

@sarah_rigdon

Adam Robbins

research

Adam Robbins works with communities in the Midwest and East Asia to achieve LGBT equality. He helped Minnesotans United for All Families achieve the first victory against an anti-marriage constitutional amendment, trained with the New Organizing Institute, and is currently working with a startup in Beijing to measure and develop the LGBT market in China.

@robbinsadam

Anastasia Savvina

Sr. Copywriter, Kaiser Permanente

editor

Anastasia is a senior copywriter by day and freelance writer in her off hours. She received a bachelor’s degree in Literature & Writing from UC San Diego and is grateful/relieved/ecstatic that she gets to use that education at work. After writing, her favorite things are traveling, backpacking, and trying new foods.

website

Alfredo Serafini

Freelance Consultant

editor

Alfredo has a diverse professional background that includes working as a consultant for information retrieval and linked data, teaching programming languages, and designing soundscapes. You can follow his various pursuits on his “designed to be unfinished” website.

@seralf website

Christina Shomaker

Founder, Dumblebee

distribution

Christina is a longtime tech nerd and founder of Dumblebee, the advice-based social network. Her never-ending curiosity has led her to work in many fields, including anthropology, photography, graphic design, museum research, and real estate.

@DumblebeeIt website

Ian Tebbutt

Senior Data Analyst

writer

Ian knows big data. He’s worked with it for more than two decades, doing everything from UK TV rating calculations to fraud protection for Australian banks to health datasets for hospitals. His team of data experts is small but powerful, known for producing good data and presenting it well.

website

Kevin Taylor

NGS Program Manager, NetApp

editor

Kevin has spent the past 15 years in Business Intelligence, focusing primarily on the effective delivery of information and insight. Downright passionate about visualization, he finds beauty in simplicity and loves reading and following the pioneers of the industry. Outside the 9-to-5, he enjoys spending time with his beautiful wife Katherine and their two beautiful twin daughters, Kaseyleigh and Kennedy.

@KevinTaylorNC

Jessica Thomas

Marketing Communications Manager

distribution, editor

Jessica develops communications plans and thought leadership platforms for clients in architecture, engineering, design, higher education, and the arts. She enjoys translating technical jargon into plain language and believes in the power of good design to communicate even the most complex information.

@speakyourdesign website

Alex Wall

Digital Marketing Strategist, Roar Media

editor

Alex Wall is a digital marketing strategist in Coral Gables, Florida, and freshly addicted to data visualization.

@alexlwall website

Beatriz Wallace

Visiting Professor, Duquesne University

project manager

A New Orleans native, Beatriz currently lives in Pittsburgh, working as a Visiting Professor at Duquesne University. She helped implement digital storytelling for children exposed to family violence in rural Missouri and is currently working with University of Missouri Professor Berkley Hudson on a project that explores photographs of the South.

@BigMuddyHeart website

Christopher Wetherill

PhD Student, Virginia Tech

technical editor

Christopher is a PhD student in Virginia Tech’s Translational Biology, Medicine, and Health program. He also enjoys cat yodeling on the side.

website

Greg Zobel

Assistant Professor, Western Oregon University

editor

Gregory Zobel lives in Oregon’s Willamette Valley. He collects books and loves budgies (better known as parakeets to the uninitiated).

@drgbz website

</div> </div>