Data 2020 - NewSQL and Mystique (Brisbane)
Dave Thomas: NewSQL - The New Universal Query Language for Everything.
Back to the Future! In 2020 many new and legacy data systems, such as Spark, Teradata, Mongo, Cassandra, Splunk, Kafka, Flink, Azure, Google, Oracle…, will support a NewSQL.
We begin with a very brief review of the current state of data engineering practice - NoSQL, Big Data, Streaming, Time Series, BASE versus ACID, Graphs; DSLs, Map Reduce, Functional Programming etc. We look at the challenges presented by the plethora of different data formats, languages/tools and eventual consistency.
What if we didn’t have to deal with the challenges of sharding and eventual consistency? We discuss the recent increased use of new distributed ACID databases such as Google Spanner, AWS AuroraDB, Azure CosmosDB etc.
What if we didn’t need to worry about different languages, APIs for programming different databases? What if we could use one language for both batch and streaming? Very recently, at the ACM SIGMOD 2019 database conference, a new emerging data language was presented which seeks to provide a solution.
We refer to this emerging standard as NewSQL, which extends SQL to deal with both streaming and batch. It removes major limitations in SQL and provides the additional capabilities to SQL needed to deal with the velocity, volume and variety of diverse data sources. In this talk, we describe the evolution and major features of the new language. We conclude with a brief discussion of the impact on data engineering, data science and data consumers.
Danial Tham: Mystique - The Fight Against Rego Plate Cloning
In the rise of AI technologies, there is no doubt that privacy is a very popular topic which normally revolves around the negative impact of AI to our privacy. However despite the norm, we at Carsales built and deployed an AI tech called Mystique which fights crimes and protects user privacy. Mystique detects a rego plate in a photo and blurs it, protecting our private seller and dealer from rego plate cloning.
Rego plate cloning “involves falsifying a genuine number plate and attaching it to another vehicle which is often the same make, model and colour. The criminal may then use the car with cloned plates to drive on tollways without consequences, drive dangerously or conduct other criminal activity, with the genuine vehicle owner often sought to pay fines or tolls.”
In this talk, we are going to share the story about how we built and delivered Mystique and more importantly what lessons we learned in deploying tech at this scale, which processes 150,000 photos a day and touches many areas of business at Carsales.
CSO Kx Systems
Dave Thomas, Chief Scientist/CSO, Kx Systems, Co-Founder and past Chairman of Bedarra Research Labs (BRL), Founder of the Agile Alliance and ACM Distinguished Engineer. Founder and past CEO of Object Technology International (OTI), becoming CEO of IBM OTI Labs after its sale to IBM. With a unique ability to see the future and translate research into competitive products, he is known for his contributions to Object Technology including IBM VisualAge and Eclipse IDEs, Smalltalk and Java virtual machines. Dave is a popular, humorous, albeit opinionated keynote speaker with an impressive breadth of business experience and technical depth. He is a thought leader in large-scale software engineering and a founding director of the Agile Alliance. With close links the R&D community Dave is an adjunct research professor at Carleton University in Canada and held past positions at UQ and QUT in Australia. He has been a business and technical advisor to many technology companies including Kx Systems. Dave is founder and chairman of the YOW! Australia and Lambda Jam conferences, and is a GOTO Conference Fellow.
AI Software Engineer
Danial is a AI software engineer in Carsales.com.au. Prior to dipping his toes in the ever evolving field of AI, Danial have experience that spans across industry and been in roles different roles like Developer, QA, and DevOps for Australia's ehealth record system MyHealthRecord. Danial is the also the primary developer of Mystique an AI tech in Carsales which protects users privacy by blurring the vehicle licence plate.
Danial has been .NET enthusiast and now most of the time he works with Python which he has a love hate relation with.