QuestionQ122

VerifiedIngesting and processing the data

You manage a Dataplex environment that contains raw and curated zones. A data engineering team uploads JSON and CSV files to a bucket asset in the curated zone, but Dataplex does not automatically discover the files. What should you do to ensure Dataplex discovers them?

Explanation

For Cloud Storage assets in a curated Dataplex zone, structured data must use Parquet, Avro, or ORC and be organized in a Hive-compatible directory layout. CSV and JSON are supported semi-structured formats for raw zones, so the files must be placed in the raw zone for Dataplex discovery. Google Cloud Dataplex Zone reference

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!