Join us at Data + AI Summit Europe – Finally Big Data, Analytics and AI converge
1 day ago, databricks

Data + AI Summit Europe will take place 17-19 November and is now open for registration!  Formerly known as the... The post Join us at Data + AI Summit Europe – Finally Big Data, Analytics and AI converge appeared first on Databricks....

Registration Open for Inaugural Data + AI Summit Europe
1 day ago, databricks

Data + AI Summit Europe will take place 17-19 November and is now open for registration!  Formerly known as the... The post Registration Open for Inaugural Data + AI Summit Europe appeared first on Databricks....

Automate Azure Databricks Platform Provisioning and Configuration
2 days ago, databricks

Table of Contents Introduction Automation options Common workflow Pre-Requisites Create Azure Resource Group and Virtual Network Provision Azure Application /... The post Automate Azure Databricks Platform Provisioning and Configuration appeared first on Databricks....

Building a Modern Risk Management Platform in Financial Services
3 days ago, databricks

This blog was collaboratively written with Databricks partner Avanade. A special thanks to Dael Williamson, Avanade CTO, for his contributions. Financial Institutions today are still struggling to keep up with the emerging risks and threats facing their business. Managing risk, especially within the banking sector, has increased in complexity over the past several years. First, […] The post Building a Modern Risk Management Platform in Financial Services appeared first on Databricks....

Easily Clone your Delta Lake for Testing, Sharing, and ML Reproducibility
3 days ago, databricks

Introducing Clones An efficient way to make copies of large datasets for testing, sharing and reproducing ML experiments  We are excited to introduce a new capability in Databricks Delta Lake – table cloning. Creating copies of tables in a data lake or data warehouse has several practical uses. However, given the volume of data in […] The post Easily Clone your Delta Lake for Testing, Sharing, and ML Reproducibility appeared first on Databricks....

Announcing Databricks Labs Terraform integration on AWS and Azure
1 week ago, databricks

We are pleased to announce integration for deploying and managing Databricks environments on Microsoft Azure and Amazon Web Services (AWS) with HashiCorp Terraform. It is a popular open source tool for creating safe and predictable cloud infrastructure across several cloud providers. With this release, our customers can manage their entire Databricks workspaces along with the […] The post Announcing Databricks Labs Terraform integration on AWS and Azure appeared first on Databricks....

Diving Deep Into the Inner Workings of the Lakehouse and Delta Lake
1 week ago, databricks

Earlier this year, Databricks wrote a blog that outlined how more and more enterprises are adopting the lakehouse pattern. The blog created a massive amount of interest from technology enthusiasts. While lots of people praised it as the next-generation data architecture, some people thought the Lakehouse is the same thing as the data lake. Recently, […] The post Diving Deep Into the Inner Workings of the Lakehouse and Delta Lake appeared first on Databricks....

It’s an ESG World and We’re Just Living in it
1 week ago, databricks

The future of finance goes hand in hand with socially responsible investing, environmental stewardship, and corporate ethics. In order to stay competitive, Financial Services Institutions (FSI) are increasingly disclosing more information about their environmental, social, and corporate governance (ESG) performance. Hence the increasing importance of ESG ratings and ESG scores to investment managers and institutional […] The post It’s an ESG World and We’re Just Living in it appeared first on Da...

An Update on Project Zen: Improving Apache Spark for Python Users
2 weeks ago, databricks

Apache Spark™ has reached its 10th anniversary with Apache Spark 3.0 which has many significant improvements and new features including but not limited to type hint support in pandas UDF, better error handling in UDFs, and Spark SQL adaptive query execution. It has grown to be one of the most successful open-source projects as the […] The post An Update on Project Zen: Improving Apache Spark for Python Users appeared first on Databricks....

Spark + AI Summit Europe is Expanding and Getting a New Name: Data + AI Summit Europe
2 weeks ago, databricks

Back in 2013, we held the first Spark Summit — a gathering of the Apache Spark™ community with leading contributors and production users sharing their wisdom. Since the first event, Spark’s success has accelerated the evolution of data science, data engineering and analytics. As the data community has expanded, we’ve evolved the content and the […] The post Spark + AI Summit Europe is Expanding and Getting a New Name: Data + AI Summit Europe appeared first on Databricks....

Databricks Unified Data Analytics Platform for AWS Gets a Major Upgrade
2 weeks ago, databricks

Now generally available an upgraded platform architecture that adds customer-managed IP access lists, customer managed-VPC, Account API, multiple workspaces per account, cluster level policies, IAM credential passthrough, and more. We are excited to announce the general availability of a major upgrade to the Databricks Unified Data Analytics Platform on Amazon Web Services  (AWS) that adds […] The post Databricks Unified Data Analytics Platform for AWS Gets a Major Upgrade appeared first on Data...

Introducing the Databricks Web Terminal
2 weeks ago, databricks

Introduction We’re excited to introduce the public preview of the Databricks Web Terminal in the 3.25 platform release. Any user with “Can Attach To” cluster permissions can now use the Web Terminal to interactively run Bash commands on the driver node of their cluster. The new Databricks web terminal provides a fully interactive shell that […] The post Introducing the Databricks Web Terminal appeared first on Databricks....

Operationalize 100 Machine Learning Models in as Little as 12 Weeks with Azure Databricks
3 weeks ago, databricks

In rapidly changing environments, Azure Databricks enables organizations to spot new trends, respond to unexpected challenges and predict new opportunities. Organizations are leveraging machine learning and artificial intelligence (AI) to derive insight and value from their data and to improve the accuracy of forecasts and predictions. Data teams are using Delta Lake to accelerate ETL […] The post Operationalize 100 Machine Learning Models in as Little as 12 Weeks with Azure Databricks appeared ...

Improving Public Health Surveillance During COVID-19 with Data Analytics and AI
3 weeks ago, databricks

As the leader of the State and Local Government business at Databricks, I get to see what governments all over the U.S. are doing to address the Novel Coronavirus and COVID-19 crisis. I am continually inspired by the work of public servants as they go about their business to save lives and address this crisis. […] The post Improving Public Health Surveillance During COVID-19 with Data Analytics and AI appeared first on Databricks....

Enabling Spark SQL DDL and DML in Delta Lake on Apache Spark 3.0
3 weeks ago, databricks

Last week, we had a fun Delta Lake 0.7.0 + Apache Spark 3.0 AMA where Burak Yavuz, Tathagata Das, and Denny Lee provided a recap of Delta Lake 0.7.0 and answered your Delta Lake questions.  The theme for this AMA was the release of Delta Lake 0.7.0 coincided with the release of Apache Spark 3.0 […] The post Enabling Spark SQL DDL and DML in Delta Lake on Apache Spark 3.0 appeared first on Databricks....

Women’s Equality Day: Cultivating Leadership, Communication and Negotiation Skills
3 weeks ago, databricks

Databricks is committed to fostering an environment that promotes equality and inclusion. For Women’s Equality Day on August 26, we invited Coco Brown, founder and CEO of Athena Alliance — a digital platform dedicated to revolutionizing leadership — to present a talk about women in leadership and the path to the boardroom. This talk highlighted […] The post Women’s Equality Day: Cultivating Leadership, Communication and Negotiation Skills appeared first on Databricks....

%tensorboard – a new way to use TensorBoard on Databricks
3 weeks ago, databricks

Introduction With the Databricks Runtime 7.2 release, we are introducing a new magic command %tensorboard. This brings the interactive TensorBoard experience Jupyter notebook users expect to their Databricks notebooks. The %tensorboard command starts a TensorBoard server and embeds the TensorBoard user interface inside the Databricks notebook for data scientists and machine learning engineers to visualize […] The post %tensorboard – a new way to use TensorBoard on Databricks appeared first on Da...

A Guide to the Databricks + AWS Cloud Data Lake Dev Day Workshop
3 weeks ago, databricks

The Databricks team has been working hard to recreate content and enhance the experience as we transition all our events to a virtual experience. And we’ve learned a thing or two about what you want to learn: integrating Databricks with AWS services. So we’re excited to present our Cloud Data Lake Dev Day Workshop in […] The post A Guide to the Databricks + AWS Cloud Data Lake Dev Day Workshop appeared first on Databricks....

Profit-Driven Retention Management with Machine Learning
3 weeks ago, databricks

Companies with the highest loyalty ratings and retention rates grew revenues 250% faster than their industry peers and delivered two to five times the shareholder returns over a 10 year period. Earning loyalty and getting the largest number of customers to stick around is something that is in the best interest of both a company […] The post Profit-Driven Retention Management with Machine Learning appeared first on Databricks....

Top 5 Reasons to Convert Your Cloud Data Lake to a Delta Lake
4 weeks ago, databricks

If you examine the agenda for any of the Spark Summits in the past five years, you will notice that there is no shortage of talks on how best to architect a data lake in the cloud using Apache Spark™ as the ETL and query engine and Apache Parquet as the preferred file format.  There […] The post Top 5 Reasons to Convert Your Cloud Data Lake to a Delta Lake appeared first on Databricks....

Next