Professional Data Engineer

By Google · Question Mode

QuestionQ48

Storing the data

You launched a new gaming app nearly three years ago. You have uploaded the previous day’s log files to separate Google BigQuery tables named in the format LOGS_yyyymmdd. You have used table wildcard functions to create daily and monthly reports across all time ranges. Recently, you found that some queries spanning long date ranges exceed the 1,000-table limit and fail. How can you resolve this issue?

Explanation

BigQuery recommends partitioning rather than date sharding. A single partitioned table stores the daily data as partitions, so date-range queries can prune the relevant partitions without referencing more than 1,000 separate sharded tables.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!