{"id":798,"date":"2014-10-08T16:15:56","date_gmt":"2014-10-08T21:15:56","guid":{"rendered":"http:\/\/www.cognitiveinfo.com\/?p=798"},"modified":"2014-10-08T16:15:56","modified_gmt":"2014-10-08T21:15:56","slug":"time-value-of-data","status":"publish","type":"post","link":"http:\/\/cognitiveinfo.com\/?p=798","title":{"rendered":"Time Value of Data"},"content":{"rendered":"<p><a href=\"http:\/\/www.cognitiveinfo.com\/wp-content\/uploads\/2014\/10\/DollarCloud.jpg\"><img fetchpriority=\"high\" decoding=\"async\" class=\"aligncenter wp-image-801 size-medium\" src=\"http:\/\/www.cognitiveinfo.com\/wp-content\/uploads\/2014\/10\/DollarCloud-300x225.jpg\" alt=\"DollarCloud\" width=\"300\" height=\"225\" \/><\/a>The concept of &#8216;Time Value&#8217; of data and the associated costs of data are a segment of the &#8216;BI Basics&#8217; session which always brings up a lot of discussion\u00a0at SQL Saturday.\u00a0 Here, I will expand on the discussion and hope you will join in below.\u00a0 This session was given at SQL Saturday in Oklahoma City and Kansas City and can be downloaded <a title=\"SQL Saturday #300 Kansas City, MO 2014\" href=\"http:\/\/www.cognitiveinfo.com\/2014\/sql-saturday-300-kansas-city-mo-2014\/\">here<\/a>.<\/p>\n<p>My background is in economics with a good bit of finance thrown in. In Economics, we often look at markets in terms of supply and demand curves.<br \/>\nIn Finance studies, the &#8216;time value of money&#8217; is a concept which is played out in countless equations. The basic concept is that a dollar today is worth more than a dollar in the future. Would you rather I give you a dollar today or a dollar tomorrow?<\/p>\n<p>My theory about the &#8216;Time Value of Data&#8217; incorporates elements of time value and then charts &#8216;value &amp; cost curves&#8217; similar to &#8216;supply &amp; demand curves&#8217;.\u00a0 This is because I needed a way to visualize the concept and share it.\u00a0 So far, the values shown are merely relative and are not based on scientific survey of business users.\u00a0\u00a0\u00a0 It is based on my experience in many industries and many companies over the past 16 years of helping companies use their data.<\/p>\n<p><strong>Time Value of Data:<\/strong><\/p>\n<p>My &#8216;Time Value&#8217; curve sort of combines those two basic theories by looking at the latency of getting data ready for business users to make business decisions. For a typical company there is a lot of value in having data about last year, last month and even quite a bit of value in yesterday&#8217;s data. Few businesses will consider it added value to have up to the minute data, so for our purposes current data has relatively less value. There are always exceptions, like monitoring a nuclear reactor but we&#8217;re talking about a typical company selling typical stuff.<\/p>\n<p><a href=\"http:\/\/www.cognitiveinfo.com\/wp-content\/uploads\/2014\/10\/BI-Basics-time-value.jpg\"><img decoding=\"async\" class=\"aligncenter wp-image-802 size-medium\" src=\"http:\/\/www.cognitiveinfo.com\/wp-content\/uploads\/2014\/10\/BI-Basics-time-value-300x225.jpg\" alt=\"BI-Basics-time-value\" width=\"300\" height=\"225\" \/><\/a><strong>Time Costs of Data:<\/strong><\/p>\n<p>The other component to look at is the &#8216;Cost of Data&#8217; based upon latency. These costs can include data storage, data cleansing, data extract, data transfer, programming, hardware, software, network infrastructure and even purchasing outside data.<\/p>\n<p>Costs of Data include but are not limited to:<\/p>\n<ul>\n<li>Storage.\u00a0 The people who say &#8216;storage is cheap&#8217; are usually not the person writing a check for a new SAN.<\/li>\n<li>Acquisition. This may include paying for data sets or subscribing to data feeds.<\/li>\n<li>ETL \u2013 Extract Transform &amp; Load<\/li>\n<li>Cleansing.\u00a0 This may include services such as cleansing addresses.<\/li>\n<li>Data Quality Initiatives<\/li>\n<li>Data Governance Initiatives<\/li>\n<li>Archiving<\/li>\n<\/ul>\n<p>To aggregate and store last year&#8217;s data pretty affordable and probably already done in most companies. Many companies companies have also built or bought some type of monthly reporting and analysis system. This shift in latency from last year to last month, usually increases the costs only slightly. To have yesterday&#8217;s data today in a form that easily usable by business users, often requires significant investment in hardware, software and programming to accumulate the data, verify it and prepare it for users. These initiatives are often data marts or data warehouses designed for reporting and analysis.<\/p>\n<p>Decreasing data latency from daily to near real time can be very expensive.<\/p>\n<p>I always stop here to mention, data values and costs can vary widely within a company, especially large enterprises.\u00a0\u00a0 The executives may highly value a daily summary of the company activity, but the production manager on a plant floor may value &#8216;up to the minute&#8217; details.\u00a0\u00a0 The former requires company wide data acquisition, while the latter may only require reporting from one system.\u00a0\u00a0 This example shows that within one company there are varying values and varying costs.\u00a0 Balancing those needs requires thorough knowledge of the requirements when designing solutions.<\/p>\n<p><strong>Sweet Spot<\/strong><\/p>\n<p>The point at which data value outweighs the costs to collect and aggregate data is the sweet spot where IT needs to deliver solutions.\u00a0 As we discuss in the BI Basics class, data warehousing is an architecture while big data is a technology, both of which can help deliver data quickly in usable form to the users.\u00a0\u00a0 Which path you take will depend on many other factors learned during your requirements gathering across your company.<\/p>\n<p>I have seen data warehouse projects which started with a target of daily refresh and then find incredible value in their data which justified the costs to get the data near real time. This shift can easily cost six-figures for a medium sized enterprise.\u00a0\u00a0 In other words, once IT delivered data to meet the sweet spot shown above with daily latency,\u00a0 the business found higher value in near real time data.\u00a0\u00a0 In economics, we would call this a demand shift.<\/p>\n<p>I hope this article will provide some discussion points for you with your data managers. \u00a0 You will find your company ends up in the spectrum somewhere between &#8220;gotta have it now&#8221;\u00a0 to &#8220;last week&#8217;s data is good enough&#8221;.\u00a0\u00a0 Either way, wishing you the best of success with your data projects.<\/p>\n<p>&nbsp;<\/p>\n<p><em>About the Author:<\/em><\/p>\n<p><span id=\"ContentPlaceHolder1_lblSpeakerBio\"><em>Allen Smith is a business intelligence and data warehouse consultant, speaker and trainer. Allen has an MBA in Computer Sciences and has worked in database services and data warehousing since 1998.\u00a0 Allen has a MCTS certification in Business Intelligence. Allen is a principal consultant at Cognitive Information Inc., a consulting firm in Edmond, OK.<\/em> <\/span><\/p>\n<p>About the graphic:\u00a0\u00a0 I couldn&#8217;t resist a graphic based on a recent <a href=\"http:\/\/dilbert.com\/strips\/comic\/2014-09-21\/\" target=\"_blank\" rel=\"noopener\">Dilbert<\/a>.<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Data has a value and a cost to the business, based on the latency of the data and the needs of the business.<\/p>\n","protected":false},"author":2,"featured_media":801,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2,7],"tags":[29,30,43,46],"class_list":["post-798","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bidw-projects","category-featured","tag-business-2","tag-business-intelligence","tag-data","tag-data-warehouse"],"_links":{"self":[{"href":"http:\/\/cognitiveinfo.com\/index.php?rest_route=\/wp\/v2\/posts\/798","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/cognitiveinfo.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/cognitiveinfo.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/cognitiveinfo.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/cognitiveinfo.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=798"}],"version-history":[{"count":0,"href":"http:\/\/cognitiveinfo.com\/index.php?rest_route=\/wp\/v2\/posts\/798\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/cognitiveinfo.com\/index.php?rest_route=\/"}],"wp:attachment":[{"href":"http:\/\/cognitiveinfo.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=798"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/cognitiveinfo.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=798"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/cognitiveinfo.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=798"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}