MSDS PYSPARK WEEK7 - Arton

0

Please login or register to vote for this query.

(click on this box to dismiss)

Gardening and Landscaping

Q&A for gardeners and landscapers

-- Change TagId in PostTags with Equivalent Word
-- If PostId in Posts from 2017 NOT in above table, then delete row


SELECT a.PostID, a.TagName
FROM (SELECT 
  a.PostID, 
  b.TagName
FROM 
  PostTags a
  INNER JOIN Tags b
ON 
  a.TagId = b.Id) a
INNER JOIN (SELECT Id AS PostID FROM Posts WHERE CreationDate Like '%2017%') b
ON a.PostID = b.PostID
ORDER BY a.PostId;
  
  

-- The CSV file can now be used in PySpark for Rapid Cluster Key-Value Pairing

-- Note that BigQuery doesn't contain the PostTags table: 
-- This is a problem because idk how Pyspark will react with Tags that look like 
-- "postgres|Java|BigQuery". Of course, you can seprate them manually in Python
-- but if we're pretending this is a very big table, that would take decades

Enter Parameters

Options:
Switch to meta site
loading Hold tight while we fetch your results