NiFi, or Niagara Files, is a flow-based programming solution developed by the NSA. It was open sourced in 2014 and is commercially supported. More than an ETL solution, it has the power to transform the way your business processes and consumes data.
Reason #1: Let NiFi do some Heavy Lifting before the data hits the database
Using stored procedures to translate and process data is a common approach. At scale, it’s one of the more costly ETL methods. Why? Because storing processing logic in a database requires the entire database to scale. Including the licenses.
Among other things, NiFi can:
- Ensure a file’s schema
- Lookup a value from another source based on an input field
- Translate to another format
- Support many input formats and methods
- Scale to your heart’s content
If you haven’t had a chance to tinker with NiFi’s functionality, it’s easy enough. Here’s a tutorial to help get you started.[Read more…] about NiFi: Three Great Reasons it Should be in Your Tech Toolkit