At PASS Summit, Microsoft made SQL Server Integration Services (SSIS) a first-class citizen within Azure Data Factory. That is no small feat. SSIS arose along with SOAP at a time when XML APIs were synonymous with web services. Extract, transform and load (ETL) is an essential part of an SSIS workflow.
But a lot has changed since the first heady days of web services. With Azure Data Factory, use of REST APIs takes prominence, along with a data pipeline that supports Hive, Pig, Hadoop and other new age data tools. In the cloud, Extract, Load, Transform (ELT) is the approach that tends to win out — a slight adjustment in focus perhaps, but representative of larger changes taking place.
Now, with Azure Data Factory Version 2 recently available as a technology preview, SSIS ETL developers will be able to map their ETL skills more easily to the cloud environment, according to Carlos Bossy, who led a PASS session on implementing BI in the cloud.
For example, some SSIS work can move to the cloud without rewriting.
“Azure data integration was somewhat of a problem until this year with SSIS changes,” said Bossy, who is senior managing partner and an architect at BI consultancy Datalere, based in Denver. Bossy showed BI developers a smorgasbord of data frameworks referring to Apache Kafka, Apache Spark and a host of others.
That is the rub in distributed data processing on the cloud. As Bossy said, “There’s so much stuff — it’s hard to tell how to use it.”
He advised BI developers to proceed at a reasonable pace, which seemed to resonate for conference attendees who have to bring in the new at the same time as they keep existing things running.
“Not everybody needs all of this,” Bossy said. For that matter, he quipped, “some people are still trying to get yesterday’s sales report out.”