DAT264x: Identifying Appliances from Energy Use Patterns
Hosted By Microsoft
You're not part of this competition. Yet...Join the competition!
Identifying Appliances from Energy Use Patterns
According to a 2017 report, the U.S. Energy Information Administration projects a 28% increase in world energy consumption by 2040. And the energy sector is a major contributor to climate change. For example, energy production and use accounts for more than 84% of U.S. greenhouse gas emissions.
Increasing the efficiency of energy consumption has benefits for consumers, providers, and the environment. With an increasing number of IoT devices coming online in the energy sector, there is more and more data that can be used to monitor and track energy consumption. Ultimately, this type of data can be used to provide consumers and businesses with recommendations on ways to save energy, lower costs, and help the planet.
In this challenge, you will use standard AI tools to identify 11 different types of appliances from their electric signatures, quantified by current and voltage measurements.
This plug load dataset contains current and voltage measurements sampled at 30 kHz from 11 different appliance types present in more than 60 households in Pittsburgh, Pennsylvania. Plug load refers to the energy used by products that are powered by means of an ordinary AC plug (i.e., plugged into an outlet). For each appliance, plug load measurements were post-processed to extract a two-second-long window of measurements of current and voltage. For some observations, the window contains both the startup transient state (turning the appliance on) as well as the steady-state operation (once the appliance is running). For others, the window only contains the steady-state operation. The observations were then transformed into two spectrograms, one for current, and one for voltage.
A spectrogram is a visual representation of the various frequencies of sound as they vary with time. The x-axis represents time (2 seconds in our case), and the y-axis represents frequency (measured in Hz). The colors indicate the amplitude of a particular frequency at a particular time (i.e., how loud it is). We're measuring amplitude in decibels, with 0 being the loudest, and -80 being the softest. So in the example spectrogram below, lower frequencies are louder than higher frequencies. Our spectrograms tend to have horizontal lines given that we are capturing appliances in their steady-state. In other words, the amplitudes of various frequencies are fairly constant over time.
Spectrograms were created using librosa, a python package for music and audio analysis. The code to generate a spectrogram looks like this:
S = librosa.feature.melspectrogram(y=obs, sr=30000) spectrogram = librosa.power_to_db(S) plt.imsave(file_path, arr=spectrogram)
Under the hood, this process:
- Takes the fourier transform of a windowed excerpt of the raw signal, in order to decompose the signal into its consistuent frequencies.
- To learn more about fourier transforms, check out this awesome tutorial by 3Blue1Brown: But what is the Fourier Transform.
- Maps the powers of the spectrum onto the mel scale. The mel scale is a perceptual scale where pitches are judged to be equal in distance from one another based on the human ear.
- Takes the logs of the power (amplitude squared) at each of the mel frequencies to convert to decibel units.
- Plots and saves the resulting image.
There is a lot of useful information encoded in these spectrograms. Now it's time to use your deep learning skills to parse out which patterns correspond to which types of appliances.
DAT264x: Identifying Appliances from Energy Use Patterns: Rules and Terms of Data Use
DrivenData Competition Rules
One account per participant
You cannot sign up to this website http://datasciencecapstone.org ("Website") from multiple accounts and therefore you cannot submit from multiple accounts.
Private sharing of code
Privately sharing code or data outside of teams is not permitted.
Public dissemination of entries
DrivenData and the competition host have the right to publicly disseminate any entries or models.
Documentation for winning solutions
Winning solutions must be documented using the provided Winning Model Documentation Template in order to be eligible for recognition and prize money if offered.
The number of submissions per day is restricted to a fixed value on a per-competition basis. Any attempt to circumvent this limit will result in disqualification.
External data is not allowed unless otherwise noted explicitly on these competition pages. Participants agree to make no attempt to use additional data or data sources not provided on these competition pages.
These are the complete, official rules for the Competition (the 'Competition Rules') and incorporate by reference the contents of the Competition Website listed above.
By downloading a dataset linked from the Competition Website, submitting an entry to this Competition, or joining a Team in this Competition, you are agreeing to be bound by these Competition Rules which constitute a binding agreement between you and DrivenData and, if applicable, any rules and restrictions that may be imposed by the Competition Sponsor.
The Competition is sponsored by the Competition Sponsor listed above and hosted on the Competition Sponsor's behalf by DrivenData, Inc ('DrivenData'). The Competition will run according to the dates listed on the Competition Website. From time to time, DrivenData may sponsor a Competition, in which case DrivenData will act as the Competition Sponsor.
Each registered individual or Team is referred to as a Participant. You may only compete using a single, unique Website account registered at the Website. Competing using more than one account per individual is a breach of these Competition Rules and Competition Sponsor reserves the right to disqualify any individual (or Team including an individual) who is found to breach these Competition Rules.
The Competition is open to all individuals over the age of 18 at the time of entry and to all validly formed legal entities that have not declared or been declared in bankruptcy.
Officers, directors, employees and advisory board members (and their immediate families and members of the same household) of the Competition Sponsor, DrivenData and their respective affiliates, subsidiaries, contractors, agents, judges and advertising and promotion agencies are not eligible to participate in the Competition.
You are not eligible to receive any Prize in the Competition if you are a resident of a country designated by the United States Treasury's Office of Foreign Assets Control.
If you meet the eligibility requirements and would like to participate, then you must first complete the registration process at the Website within the Competition Period. After you complete the registration process, you will receive access to the Data (described on the Competition page) that will enable you to develop and submit one or more Entries (as defined below). All Entries must be received during the Competition Period. To register, visit the Website and follow the onscreen instructions to complete and submit your registration. All of the registration information that you provide is collectively referred to as your "Account". (If you have already created an Account at the Website, enter your user name and password and follow the on-screen instructions).
After you register individually, you may join a team (a "Team"), only if expressly permitted by these rules below, but you may register only once. If you register for the Competition more than once, you will be, and the remainder of your Team may be, disqualified in Sponsor's sole discretion.
You acknowledge and agree that you are solely responsible for abiding by your employer's policies regarding participation in the Competition. Sponsor disclaims any and all liability or responsibility for disputes arising between an employee and employer related to this Competition.
Once you have completed the registration process, you or your Team will be provided with access to the Data that you use to develop your Entries.
BY ENTERING THE COMPETITION YOU ACCEPT THE CONDITIONS STATED IN THESE OFFICIAL RULES INCLUDING THE RULES ON THE COMPETITION PAGE, AGREE TO BE BOUND BY THE DECISIONS OF THE JUDGES AND WARRANT THAT YOU ARE ELIGIBLE TO PARTICIPATE IN THE COMPETITION. IF YOU DO NOT ACCEPT ALL OF THESE OFFICIAL RULES, THEN PLEASE DO NOT REGISTER FOR THE COMPETITION. WE RECOMMEND THAT YOU PRINT OUT A COPY OF THESE OFFICIAL RULES FOR YOUR FUTURE REFERENCE.
FORMING A TEAM. If DrivenData enables team functionality for this Competition, multiple individuals or entities may collaborate as a team ("Team"). You may not participate on more than one Team. Each Team member must be a single individual operating a separate Website account. You must register individually for the Competition before joining a Team. You must confirm your Team membership to make it official by responding to the Team notification message which will be sent to your Account.
TEAM PRIZES. If a Team wins a monetary Prize, Competition Sponsor will allocate the Prize money in even shares between the Team members unless the Team unanimously contacts DrivenData within three business days following the Submission deadline to request an alternative prize distribution.
Each Entry must be uploaded to the Website in the manner and format specified on the Website. All Entries must be received during the Competition Period. An "Entry" is the data submitted in the manner and format specified on the Website via the Entry form. The number of Entries a Participant may submit during each calendar day of the Competition Period will be displayed on the Competition page. Sponsor reserves the right to request that a Participant submit the algorithm associated with an Entry to Sponsor. If the Competition is a multi-stage Competition with temporally separate training data and/or leaderboard data, one or more valid Entries must be submitted and selected during each stage of the Competition in the manner described on the Website.
USER SUBMISSIONS MUST IN THEIR ENTIRETY COMPLY WITH ALL APPLICABLE FEDERAL, STATE, LOCAL, AND INTERNATIONAL LAWS AND REGULATIONS. WITHOUT LIMITING THE FOREGOING, IN ORDER TO BE ELIGIBLE, YOU WARRANT THAT (UNLESS OTHERWISE SPECIFIED IN THE PROBLEM STATEMENT) YOUR SUBMISSION FOR THIS COMPETITION:
- Does not include or anticipate the inclusion of any content that is in violation of or infringes third party intellectual property rights including, but not limited to copyrights, including music copyrights, patents, trade secrets, trademarks, and rights of publicity or privacy.
- Is free and clear of all liens, claims, encumbrances or demands of any third parties.
- Does not include or anticipate the inclusion of any unsuitable or offensive content, including nudity, sexually explicit, disparaging, libelous or other inappropriate content.
- Has not been entered in previous challenges, or won previous awards.
- Has not been published or distributed previously in any media.
- Is suitable for a general audience.
- Does not contain any claims that are not and cannot be substantiated or that would be false and/or misleading to a reasonable consumer.
Selection of winners
This Competition is a challenge of skill and the final results are determined solely by leaderboard ranking on the private leaderboard (subject to compliance with these Competition Rules). Participants' scores and ranks on the Competition Website at any given stage of the Competition will be based on the evaluation metric described on the Competition Website, as determined by applying the predictions in the Submission to the ground truth of a validation dataset whose instances were a fixed set sampled from the Data.
The evaluation metric used for scoring and ranking Submissions will be displayed on the Competition Website.
Any prize awards are subject to verification of eligibility and compliance with these Competition Rules. All decisions of DrivenData, the Competition Sponsor and/or judges will be final and binding on all matters relating to this Competition. DrivenData and Competition Sponsor reserves the right to examine the Submission and any associated code or documentation for compliance with these Competition Rules. In the event that the Submission demonstrates a breach of these Competition Rules, Competition Sponsor may at its discretion take either of the following actions:
- disqualify your Submission(s); or
- require that you remediate within one week all issues identified in your Submission(s) (including, without limitation, the resolution of license conflicts, the fulfillment of all obligations required by software licenses, and the removal of any software that violates the software restrictions).
A tie between two or more valid and identically ranked submissions will be resolved in favor of the tied submission that was submitted first.
A Participant may decline to be nominated as a Winner by notifying DrivenData directly within one week following the Competition deadline, in which case the declining Participant forgoes any prize or other features associated with winning the Competition. DrivenData reserves the right to disqualify a Participant who so declines at DrivenData's sole discretion if DrivenData deems disqualification appropriate.
Prizes and conditions
There are no prizes being offered in association with challenges hosted on datasciencecapstone.org.
CHEATING. Participating using more than one Website account is deemed cheating and, if discovered, will result in disqualification from the Competition and any other affected Competitions and may result in banning or deactivation of affected Website accounts.
DrivenData reserves the right to request information associated with our investigation of suspected cheating. Failure to respond to these requests (including failure to furnish the requested information) within ten (10) days is grounds for disqualification.
Data use and code sharing
DATA. "Data" means the Data or Datasets linked from the Competition Website for the purpose of use by Participants in the Competition. For the avoidance of doubt, Data is deemed for the purpose of these Competition Rules to include any prototype or executable code provided to Participants by DrivenData or Competition Sponsor via the Website. Participants must use the Data only as permitted by these Competition Rules and any associated data use rules specified on the Competition Website.
Unless otherwise permitted by the terms of the Competition Website, Participants must use the Data solely for the purpose and duration of the Competition, including but not limited to reading and learning from the Data, analyzing the Data, modifying the Data and generally preparing your Submission and any underlying models and participating in forum discussions on the Website. Participants agree to use suitable measures to prevent persons who have not formally agreed to these Competition Rules from gaining access to the Data and agree not to transmit, duplicate, publish, redistribute or otherwise provide or make available the Data to any party not participating in the Competition. Participants agree to notify DrivenData immediately upon learning of any possible unauthorized transmission or unauthorized access of the Data and agree to work with DrivenData to rectify any unauthorized transmission. Participants agree that participation in the Competition shall not be construed as having or being granted a license (expressly, by implication, estoppel, or otherwise) under, or any right of ownership in, any of the Data.
EXTERNAL DATA. Unless otherwise expressly stated on the Competition Website, Participants must not use data other than the Data to develop and test their models and Submissions. Competition Sponsor reserves the right in its sole discretion to disqualify any Participant who Competition Sponsor discovers has undertaken or attempted to undertake the use of data other than the Data, or who uses the Data other than as permitted according to the Competition Website and in these Competition Rules, in the course of the Competition.
CODE SHARING. Participants are prohibited from privately sharing source or executable code developed in connection with or based upon the Data, and any such sharing is a breach of these Competition Rules and may result in disqualification.
Participant warranties and obligations
By registering, you agree that (a) your Account is complete, correct and accurate and (b) your registration may be rejected or terminated and all Entries submitted by you and/or your Team may be disqualified if any of the information in your Account is (or Competition Sponsor has reasonable grounds to believe it is) incomplete, incorrect or inaccurate. You are solely responsible for your Account. All registration information is deemed collected in the United States.
Participation is subject to all federal, state and local laws and regulations. Void where prohibited or restricted by law. You are responsible for checking applicable laws and regulations in your jurisdiction before participating in the Competition to make sure that your participation is legal. You are responsible for all taxes and reporting related to any award that you may receive as part of the Competition. You are responsible for abiding by your employer's policies regarding participation in the Competition. Competition Sponsor disclaims any and all liability or responsibility for disputes arising between you and your employer related to this Competition.
Each Participant is solely responsible for all equipment, including but not necessarily limited to a computer and internet connection necessary to access the Website and to develop and upload any Submission, and any telephone, data, hosting or other service fees associated with such access, as well as all costs incurred by or behalf of the Entrant in participating in the Competition.
By entering a Submission, you represent and warrant that all information you enter on the Website is true and complete to the best of your knowledge, that you have the right and authority to make the Submission (including any underlying code and model) on your own behalf or on behalf of the persons and entities that you specify within the Submission, and that your Submission:
- complies with all applicable federal, state, local and international laws and regulations;
- is your own original work, or is used by permission, in which case full and proper credit and identify is given and the third party contributions are clearly identified within your Submission;
- does not contain confidential information or trade secrets and is not the subject of a registered patent or pending patent application;
- does not violate or infringe upon the patent rights, industrial design rights, copyrights, trademarks, rights of privacy, publicity or other intellectual property or other rights of any person or entity;
- does not contain malicious code, such as viruses, timebombs, cancelbots, worms, Trojan horses or other potentially harmful programs or other material or information;
- does not and will not violate any applicable law, statute, ordinance, rule or regulation;
- does not trigger any reporting or royalty obligation to any third party; and
- was not previously published and has not won any other prize/award.
A breach of any of these warranties will result in the corresponding Submission being invalid.
Confidential Information Defined. As used in this Section, "Confidential Information" means the Competition Sponsor Materials, and any information provided by DrivenData hereunder, whether of a technical, business, or other nature (including, without limitation, information relating to a party's technology, software, products, services, designs, methodologies, business plans, finances, marketing plans, Sponsors, prospects, or other affairs), that is received by Participant.
Exclusions. Participant's obligations under this Section as to the disclosing party's Confidential Information does not include any information that Participant can document: (a) was known to Participant prior to receiving the same from the disclosing party in connection with this Agreement; (b) is independently developed by Participant without use of or reference to the Confidential Information of the disclosing party; (c) is acquired by Participant from another source without restriction as to use or disclosure; or (d) is or becomes part of the public domain through no fault or action of Participant.
Nondisclosure. In consideration of the terms and conditions of this Agreement, and for other good and valuable consideration, the receipt and sufficiency of which is acknowledged, during and after the term of this Agreement Participant will: (a) use the disclosing party's Confidential Information solely for the purpose for which it is provided; (b) not disclose the disclosing party's Confidential Information to a third party unless the third party must access the Confidential Information to perform in accordance with this Agreement and the third party has executed a written agreement that contains terms that are substantially similar to the terms contained in this Section; and (c) maintain the secrecy of, and protect from unauthorized use and disclosure, the disclosing party's Confidential Information to the same extent (but using no less than a reasonable degree of care) that Participant protects its own Confidential Information of a similar nature. If Participant is required by law or by any governmental or semi-governmental agency or court to disclose the disclosing party's Confidential Information or the terms of this Agreement Participant must give prompt written notice of such requirement to the disclosing party before such disclosure and assist the disclosing party in attempting to obtain an order protecting the Confidential Information from public disclosure.
Return of Information. Upon request, Participant will destroy or deliver to the disclosing party all Confidential Information Participant may have in its possession or control.
Injunctive Relief. Participant acknowledges that any violation or threatened violation of this Section may cause irreparable injury to the other party, entitling the disclosing party to obtain injunctive relief in addition to all legal remedies.
Equipment and costs of participation
Each Participant is solely responsible for all equipment, including a computer and modem, necessary to establish a connection to the World Wide Web, access to the World Wide Web and any telephone, data, hosting or other service fees associated with such access, as well as all costs incurred by or behalf of the Participant in participating in the Competition.
Delivery and receipt of entries
Sponsor is not responsible for (a) late, lost, stolen, damaged, garbled, incomplete, incorrect or misdirected Entries or other communications, (b) errors, omissions, interruptions, deletions, defects, or delays in operations or transmission of information, in each case whether arising by way of technical or other failures or malfunctions of computer hardware, software, communications devices, or transmission lines, or (c) data corruption, theft, destruction, unauthorized access to or alteration of Entry materials, loss or otherwise. Sponsor is not responsible for electronic communications or emails which are undeliverable as a result of any form of active or passive filtering of any kind, or insufficient space in any email account to receive email messages. Sponsor disclaims any liability for damage to any computer system resulting from participation in, or accessing or downloading information in connection with, the Competition.
Reservation of rights
DrivenData, in its sole discretion, reserves the right to disqualify any person tampering with the entry process, the operation of the Web site, the competition process, or is otherwise in violation of the rules. DrivenData reserves the right at any time to disqualify a Submission from a Competition where, acting in good faith, it believes there are reasonable grounds to warrant disqualification. For example, DrivenData determination that the Submission does not provide the functionality described or required, or the Submission appears to be purposely designed to circumvent these Rules or the spirit of the Competition would be grounds for disqualification. DrivenData reserves the right to cancel, terminate or modify the competition if it is not capable of completion as planned for any reason, including infection by computer virus, bugs, tampering, unauthorized intervention or technical failures of any sort.
By entering the Competition, you release and discharge the Competition Entities from any liability whatsoever in connection with the Competition or with the acceptance, possession, use or misuse of any prize including, without limitation, legal claims, costs, injuries, losses or damages, demands or actions of any kind (including, without limitation: personal injuries; death; damage to, loss or destruction of property; rights of publicity or privacy; and defamation or portrayal in a false light). The Competition Entities will not be responsible for typographical, printing or other inadvertent errors in these Official Rules or in other materials relating to the Competition. Additionally, you hereby agree to indemnify the Competition Entities from any and all losses, damages, costs, expenses, rights, claims, demands and actions (including attorney's fees and expenses for litigation and settlement), which may be brought against any one or more of them by anyone claiming to have suffered loss or damage as a result of your participation in the Competition.
Limitations of liability
By participating in the Competition, each Participant agrees that: (1) any and all disputes, claims, and causes of action arising out of or in connection with the Competition, or any prize awarded, shall be resolved individually without resort to any form of class action; (2) any claims, judgments and awards shall be limited to actual out-of-pocket costs incurred, including costs associated with entering the Competition, but in no event attorney's fees; and (3) under no circumstances will a Participant be permitted to obtain any award for, and each Participant hereby waives all rights to claim, punitive, incidental or consequential damages and any and all rights to have damages multiplied or otherwise increased and any other damages, other than damages for actual out-of-pocket expenses.
All federal, state, and local laws apply. DrivenData, Inc. and the employees, officers, directors, shareholders, agents, representatives, affiliates, subsidiaries, and advertising, promotional and legal advisors are not responsible and shall not be liable for: (i) late, lost, delayed, damaged, misdirected, incomplete, illegible, or unintelligible entries; (ii) telephone, electronic, hardware, or software program, network, Internet, or computer malfunctions, failures, or difficulties; (iii) errors in transmission; (iv) any condition caused by events beyond the control of DrivenData, Inc. that may cause the contest to be disrupted or corrupted; (v) any injuries, losses, or damages of any kind caused by a prize or resulting from acceptance, possession, or use of a prize, or from participation in the contest; or (vi) any printing or typographical errors in any materials associated with the contest.
Disclaimer of Warranties
You understand that we cannot and do not guarantee or warrant that files available for downloading from the internet or the Website will be free of viruses or other destructive code. You are responsible for implementing sufficient procedures and checkpoints to satisfy your particular requirements for anti-virus protection and accuracy of data input and output, and for maintaining a means external to our site for any reconstruction of any lost data. WE WILL NOT BE LIABLE FOR ANY LOSS OR DAMAGE CAUSED BY A DISTRIBUTED DENIAL-OF-SERVICE ATTACK, VIRUSES OR OTHER TECHNOLOGICALLY HARMFUL MATERIAL THAT MAY INFECT YOUR COMPUTER EQUIPMENT, COMPUTER PROGRAMS, DATA OR OTHER PROPRIETARY MATERIAL DUE TO YOUR USE OF THE WEBSITE OR ANY SERVICES OR ITEMS OBTAINED THROUGH THE WEBSITE OR TO YOUR DOWNLOADING OF ANY MATERIAL POSTED ON IT, OR ON ANY WEBSITE LINKED TO IT.
YOUR USE OF THE WEBSITE, ITS CONTENT AND ANY SERVICES OR ITEMS OBTAINED THROUGH THE WEBSITE IS AT YOUR OWN RISK. THE WEBSITE, ITS CONTENT AND ANY SERVICES OR ITEMS OBTAINED THROUGH THE WEBSITE ARE PROVIDED ON AN "AS IS" AND "AS AVAILABLE" BASIS, WITHOUT ANY WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED. NEITHER THE COMPANY NOR ANY PERSON ASSOCIATED WITH THE COMPANY MAKES ANY WARRANTY OR REPRESENTATION WITH RESPECT TO THE COMPLETENESS, SECURITY, RELIABILITY, QUALITY, ACCURACY OR AVAILABILITY OF THE WEBSITE. WITHOUT LIMITING THE FOREGOING, NEITHER THE COMPANY NOR ANYONE ASSOCIATED WITH THE COMPANY REPRESENTS OR WARRANTS THAT THE WEBSITE, ITS CONTENT OR ANY SERVICES OR ITEMS OBTAINED THROUGH THE WEBSITE WILL BE ACCURATE, RELIABLE, ERROR-FREE OR UNINTERRUPTED, THAT DEFECTS WILL BE CORRECTED, THAT OUR SITE OR THE SERVER THAT MAKES IT AVAILABLE ARE FREE OF VIRUSES OR OTHER HARMFUL COMPONENTS OR THAT THE WEBSITE OR ANY SERVICES OR ITEMS OBTAINED THROUGH THE WEBSITE WILL OTHERWISE MEET YOUR NEEDS OR EXPECTATIONS.
THE COMPANY HEREBY DISCLAIMS ALL WARRANTIES OF ANY KIND, WHETHER EXPRESS OR IMPLIED, STATUTORY OR OTHERWISE, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF MERCHANTABILITY, NON-INFRINGEMENT AND FITNESS FOR PARTICULAR PURPOSE. THE FOREGOING DOES NOT AFFECT ANY WARRANTIES WHICH CANNOT BE EXCLUDED OR LIMITED UNDER APPLICABLE LAW.
Governing Law and Jurisdiction
DrivenData shall be the sole interpreter of these Official Rules.
All matters relating to the Official Rules or the Competition and any dispute or claim arising therefrom or related thereto (in each case, including non-contractual disputes or claims), shall be governed by and construed in accordance with the internal laws of the Commonwealth of Massachusetts without giving effect to any choice or conflict of law provision or rule (whether of the Commonwealth of Massachusetts or any other jurisdiction).
The invalidity or unenforceability of any provision of these Official Rules shall not affect the validity or enforceability of any other provision. In the event that any provision is determined to be invalid or otherwise unenforceable or illegal, these Official Rules shall otherwise remain in effect and be construed in accordance with their terms as if the invalid or illegal provision was not contained herein.