pedram @pdrmnvd Twitter profile

Pinned Tweet

pedram

1 year

@josh_wills @_abhisivasailam @ananthdurai @JohnKutay all of data engineering is duct tape around the impedance mismatch between software engineering and analytics

7

25

159

Last Seen Profiles

@LukeTyr

@Baaltiel

@p_hiwarkar

@TheDream55_

@Stones_76

@AubrynDavis

@Far_Warz

@keithernesto

@LondonLuke

@pen_joanne

@ThomsonRadio

@MinnaMattsson

@BadouTetra

@JeanTerlier

@iLukacs

@ArbikieUSA

@ReneeMontgomery

@CMcGinnKES

@HinataLive

@thedestroya__

@IamEmmlink

@JeanPerdriau

@dsilva_mma

@Kirstals_

@rasijallow09

@ManiatiGeorge

@kohanesbbg

@jordy_sevitz

@Larry_TheRock

@thecatbnny

@NicolasARGCH

@Klimperman

@nzowwc

@JC_C_A

@Gol_D_rukia

@Generich_Offi

pedram

@pdrmnvd

7 months

i fucking called it. ads team begging for worse search results so that the ads team can hit their goals this year

pedram

@pdrmnvd

9 months

my newest conspiracy theory is that google is filtering b2b emails to encourage greater adwords spend

5

7

394

107

1K

7K

pedram

@pdrmnvd

27 days

all roads lead to rome

19

512

5K

pedram

@pdrmnvd

10 months

A classic from back in the day

36

325

4K

pedram

@pdrmnvd

14 days

how a data scientist looks at you after you ask him to write just one single function instead of giving you a 933 line unparametrized jupyter notebook with no requirements.txt file

22

225

4K

pedram

@pdrmnvd

2 months

the white house cabinet

tesfa⚡️

@shenmutz

2 months

It’s now illegal to NOT do the Rust re-write

9

44

518

17

298

3K

pedram

@pdrmnvd

20 days

worst part of living in the bay area is you’ll be having a perfectly beautiful dinner with your wife and then the table next to you will start talking about mongodb

30

135

3K

pedram

@pdrmnvd

2 months

Finally found a way to read Python stack traces.

29

126

2K

pedram

@pdrmnvd

1 year

pour one out for the analyst who discovered this

35

78

2K

pedram

@pdrmnvd

3 months

Creating an API key for Gemini vs OpenAI.

65

93

2K

pedram

@pdrmnvd

9 months

Built a free, open-source pipeline that fetches data from public APIs, ingests from postgres database, reads 17 million row CSV files, writes parquets , reads into DuckDB, aggregates, joins, sorts, runs dbt and completes on my laptop in a minute, with materialization and…

46

145

2K

pedram

@pdrmnvd

1 month

@coldhealing blink 182 crawled so she could walk

12

1K

pedram

@pdrmnvd

2 years

every friggin time

19

181

1K

pedram

@pdrmnvd

1 month

we could never build this today ….we don’t know how

27

144

1K

pedram

@pdrmnvd

3 years

18

119

1K

pedram

@pdrmnvd

2 years

people on linked in be like my new years resolution is to found two companies and learn five languages and people on twitter be like imma eat this whole cheesecake and pass out on the couch no one wake me until 2023

5

99

1K

pedram

@pdrmnvd

5 months

hard to believe this was all a year ago. life comes at you fast

18

45

867

pedram

@pdrmnvd

7 months

@enlightenedcoop it should never have been written. this is the kind of stuff you say in a meeting

5

4

733

pedram

@pdrmnvd

1 year

The Database Alignment Chart

37

80

715

pedram

@pdrmnvd

2 months

@JustJake immediately followed by “who wrote this garbage” “oh it was me”

2

14

606

pedram

@pdrmnvd

10 months

@chrisalbon pyenv + pyenv virtualenv

10

1

548

pedram

@pdrmnvd

8 months

@depthsofwiki “While Duchamp asserted that his readymades were done without any specific reason, art critics contend that the piece has sexual undertones of a Freudian nature” Duchamp: i thought it looked cool Art Critics: he was trying to fuck it

2

17

531

pedram

@pdrmnvd

10 months

A simple, fun example to refute the common story that ML can interpolate but not extrapolate: Red dots are training points. Green point is true data. Blue curve is best fit. Notice how it correctly predicts far outside the training distribution!

Phillip Isola

@phillip_isola

10 months

A simple, fun example to refute the common story that ML can interpolate but not extrapolate: Black dots are training points. Green curve is true data generating function. Blue curve is best fit. Notice how it correctly predicts far outside the training distribution! 1/3

70

139

1K

13

20

482

pedram

@pdrmnvd

1 year

Best data stack I’m seeing in new startups: postgres database ec2 instance named prod-do-not-touch for hosting SQL Alchemy for ORM ssh tunnel into prod as an admin panel raw sql on a postgres replica for BI

Max Musing

@MaxMusing

1 year

Best data stack I’m seeing in new startups: @PostgreSQL database @Supabase for hosting @prisma as an ORM @Basedash as an admin panel @_hex_tech for BI

4

11

107

17

24

452

pedram

@pdrmnvd

2 years

now that we’ve killed data science, what is the next sexiest job of the century we can destroy?

40

27

415

pedram

@pdrmnvd

28 days

@chrisalbon but they look so cool chris

1

0

419

pedram

@pdrmnvd

5 months

when i do linear regression in sql

11

22

404

pedram

@pdrmnvd

9 months

my newest conspiracy theory is that google is filtering b2b emails to encourage greater adwords spend

5

7

394

pedram

@pdrmnvd

2 years

This was the hardest post I've ever written, and it comes from a place of love. but, we need to talk about dbt.

37

23

360

pedram

@pdrmnvd

1 year

me explaining to the second data hire how i built the data pipelines at the startup i’ve worked at for 2 years

5

20

347

pedram

@pdrmnvd

1 year

@LeaVerou @__apf__ @AnnaDBrandt oh no how will they afford a chocolate bar without profiting off cleaning fees

2

0

326

pedram

@pdrmnvd

2 years

when someone DMs me on slack with a “quick question”

5

41

315

pedram

@pdrmnvd

7 months

@UrbaneUrban you're not at all. a lot of companies send emails, newsletters, outreach etc to other companies in order to raise awareness about their product. ive noticed over the past few years more and more of these are getting marked as spam. they're not the viagra/scam/crypto variety of…

14

5

298

pedram

@pdrmnvd

10 months

3 years ago I started playing around with a little project called @dagster . It was a breath of fresh air after having spent so much time debugging data pipelines. And now, 3 years later, I'm happy to be able to say that I'm now part of the Dagster team! 🥳

20

5

299

pedram

@pdrmnvd

4 months

the one data engineer at a startup watching the 20 software engineers review each others PRs

4

9

292

pedram

@pdrmnvd

12 days

day in the life of a data engineer

12

26

287

pedram

@pdrmnvd

7 months

stakeholders when they don't use the dashboard they requested after you spent 3 weeks building it for them

9

18

282

pedram

@pdrmnvd

3 years

can we stop shitting on zillow data scientists and start shitting on zillow executives thanks

7

12

279

pedram

@pdrmnvd

2 years

Asking "what problem are we trying to solve here" any time anyone asks me to do something I don't want to do.

13

22

271

pedram

@pdrmnvd

9 months

Being a data engineer is just saying “we could use duckdb” over and over again until someone listens

11

25

270

pedram

@pdrmnvd

9 months

when you just hired a new analyst

3

13

267

pedram

@pdrmnvd

3 months

how data engineers look at you when you ask them if they looked at the data

6

15

266

pedram

@pdrmnvd

8 months

engineers will literally invent large language models so they don’t have to write one line of sql

7

19

263

pedram

@pdrmnvd

2 years

don’t ask people “what do you do?” at parties ask them: - is this pipeline idempotent - how do you handle slowly changing dimensions - do you support column level lineage

12

20

265

pedram

@pdrmnvd

2 years

A data engineer walks into a bar. The bar is on fire. Everyone is screaming. A data scientist walks into a bar. Sits down. Waits. Waits. Waits. Waits. A data analyst walks out of the bar. A head of data walks into a bar. He goes on twitter: Why The Modern Drinking Establishmen

6

27

265

pedram

@pdrmnvd

2 years

“hey qq can you pull some numbers for the board deck tomorrow”

9

17

262

pedram

@pdrmnvd

5 months

For every ten likes this gets, I will make this data scientist's job less sexy

7

8

251

pedram

@pdrmnvd

1 year

when you ask a data team to build your financial reporting in the warehouse

14

7

250

pedram

@pdrmnvd

1 year

help

10

37

239

pedram

@pdrmnvd

2 years

putting marketing in charge of marketing analytics is like putting my dog in charge of how much food goes in her food bowl

6

13

235

pedram

@pdrmnvd

2 years

I bring you the ultimate map of the Modern Data Ecosystem. So many great category leaders here, amazing to see how this space has grown over the years 🔥🚀

Josh Devlin

@JayPeeDevlin

2 years

I love when folks draw maps of the modern data ecosystem just to position themselves as either a category leader or category member.

5

1

74

31

21

232

pedram

@pdrmnvd

1 year

why is this a universal experience?

25

9

225

pedram

@pdrmnvd

2 years

when you enable self-serve analytics

1

14

211

pedram

@pdrmnvd

10 months

@hassankhan is he describing SF or just the marina

2

0

201

pedram

@pdrmnvd

5 months

match_recognize is incredible. a 40 million row event table. want to find all groups that had at 5 successive days of a particular event completed in 13 seconds on a small SF warehouse.

7

9

203

pedram

@pdrmnvd

1 year

LEFT JOIN RIGHT JOIN INNER JOIN OUTER JOIN CROSS JOIN how about you smoke a JOINt and relax

12

17

199

pedram

@pdrmnvd

1 year

therapist: SQL can't hurt it's just a programming language SQL: Queries in UNION ALL have mismatched column count; query 1 has 50 columns, query 2 has 48 columns

12

11

195

pedram

@pdrmnvd

2 years

Ok math nerds, let’s say i wanted to learn a pure math discipline for fun, not profit. outside of hs calc and algebra and some 200-level stats courses i haven’t touched math. Where do I start? my goals are to learn something fun and beautiful and i don’t care about applicability.

122

18

187

pedram

@pdrmnvd

2 years

It's easy to confuse the cause of complexity in data systems as being from the volume of data, especially when Big Data was popular, but the real complexity in data is from the number of systems that interact with data, not the volume.

14

24

188

pedram

@pdrmnvd

3 years

data person: given these set of assumptions, and for at least one segment, we believe it is possible that we’re seeing at least a marginal effect that can potentially be explained by our experiment under the following conditions sales person: DUDE WE ABSOLUTELY CRUSHED IT

5

19

184

pedram

@pdrmnvd

1 year

@dadiomov so crazy

2

7

180

pedram

@pdrmnvd

2 years

Some professional news: today is my last day with Hightouch! Watching HT grow from 9 to over 90 has been incredible, I’ve been lucky to work with some of my favorite people, and I’ve learned so much in my time there. Up next? spending a few days camping in Big Sur! 🏕

20

1

179

pedram

@pdrmnvd

1 year

Tableau is dead. If you’re in the BI space, there’s a huge opportunity to come in and win Tableau’s customers.

Will Perkins

@_gringuinho_

1 year

This is a problem. Companies to not rely solely on data in @Salesforce . Genie is neat. I'll never use it because I don't use 360. I'll never use other integration features. @Tableau is a core product. @Benioff , you bought a culture with the product. We don't feel the #Ohana .

19

9

140

20

179

pedram

@pdrmnvd

2 years

this is uh..something.

32

12

176

pedram

@pdrmnvd

2 months

@AlexNoonan6 it legit works really well, learned it from the great @vboykis here it is without no yapping

3

2

180

pedram

@pdrmnvd

4 months

we need to put an end to this madness

7

25

179

pedram

@pdrmnvd

27 days

Society if Data Engineers actually looked at the data

14

13

175

pedram

@pdrmnvd

1 year

ms sql server implies the existence of a mr sql server

4

17

173

pedram

@pdrmnvd

5 months

I can't believe I never shared this incredible 10-part in-depth series by @elliot_j_g on Python: Everything from Packages, to type hinting, factory patterns, and even CI/CD.

Python Packages: a Primer for Data People (part 1 of 2) | Dagster Blog

The foundation of a solid Python project is mastering modules, packages and imports.

dagster.io

2

25

171

pedram

@pdrmnvd

10 months

@dril i literally do not know how else to cancel this, i don’t pay for it

11

1

157

pedram

@pdrmnvd

1 year

Have we figured out how to measure the value of a data team yet?

35

11

164

pedram

@pdrmnvd

2 years

honestly no greater joy in work than managing a team of people who are better at their jobs than you'll ever be

2

6

159

pedram

@pdrmnvd

2 years

when people ask me what data science is i show them this

2

16

157

pedram

@pdrmnvd

3 months

day in the life of a data scientist

9

16

159

pedram

@pdrmnvd

3 years

In one sentence give an analyst a heart attack, I’ll go first: “hey qq can you pull some numbers for me quickly i have a board meeting in five mins”

14

10

157

pedram

@pdrmnvd

2 years

i propose the data analytics analyst engineer job title to solve all our problems.

29

9

155

pedram

@pdrmnvd

4 months

pov you’re a data engineer logging into salesforce for the first time

11

9

155

pedram

@pdrmnvd

1 year

"data people are obsessed with tools" yea because tools are fun, do you think I'm doing all this just for the stakeholders?

9

16

156

pedram

@pdrmnvd

1 year

MRR reporting is easy. "Just" need to understand: changing plans, changing billing periods, subscription migrations, new product codes, that one time on April 27, 2017 when a manual override was done, old SQL that somehow works and no one understands, backfills, contract data in…

11

17

153

pedram

@pdrmnvd

2 years

who is the absolute king making 300k a year as analytics engineer

11

2

150

pedram

@pdrmnvd

10 months

@CarsRuinedCity who wants to live next to a stadium put your hand up

62

0

148

pedram

@pdrmnvd

5 months

@nayshins it’s just search, how hard could it be?

3

0

149

pedram

@pdrmnvd

2 years

✨Sad news✨I am no longer unemployed. I've started consulting, working with data companies on their product & marketing strategies. I also help companies with data infra. If this sounds like something you'd be interested in, reach out, would love to hear from you!

15

3

152

pedram

@pdrmnvd

2 months

pedram

@pdrmnvd

9 months

@imightbemary software engineers will literally invent another dataframe framework before writing one line of sql

5

6

63

6

7

152

pedram

@pdrmnvd

9 months

life is pain

17

0

149

pedram

@pdrmnvd

9 months

couldn’t be me

3

14

150

pedram

@pdrmnvd

6 months

the openai board trying to comfort ilya after he overplayed his hand

4

16

147

pedram

@pdrmnvd

1 year

instead of sending messages like this in slack consider sending messages like this.

26

6

148

pedram

@pdrmnvd

2 years

Some people, when confronted with a data problem, think "I know, I'll use data contracts." Now they have twenty problem

12

8

144

pedram

@pdrmnvd

2 years

Been thinking a lot about how lonely the one-person-data-team can be. I want to give back: if you’re the only data person at a startup and want free mentoring, DM me. I have 3 slots open for monthly checkins, just have a short form to help me sort through requests.

5

13

143

pedram

@pdrmnvd

1 year

how hard could it be?

15

8

144

pedram

@pdrmnvd

1 year

when you ask an engineer to document a feature

1

10

144

pedram

@pdrmnvd

3 months

we do not deserve this

Charlie Marsh

@charliermarsh

3 months

Announcing uv: an extremely fast Python package installer and resolver, written in Rust. uv is designed as a drop-in alternative to pip, pip-tools, and virtualenv. With a warm cache, uv installs are near-instant. Here, it's > 75x faster than pip and pip-tools.

197

1K

7K

9

2

140

pedram

@pdrmnvd

2 months

born too early to explore space born too late to explore earth born just in time to be a digital marketing specialist migrating to google analytics 4

1

10

142

pedram

@pdrmnvd

2 years

3

12

141

pedram

@pdrmnvd

1 year

data people: upon this initial analysis, given the data we have, it does appear that there is perhaps a correlation between these two outcomes that warrants a deeper investigation into causality. sales people: BRO WE ABSOLUTELY CRUSHED IT THIS YEAR, LETS GOOOOOO 🚀🔥!!!!!!!!!!!

4

16

138

pedram

@pdrmnvd

3 months

my hot take: At around 4-5 YOE you should be almost exclusively focusing on stakeholder management and business outcomes even to the detriment of technical ability to progress your career in data.

18

5

137

pedram

@pdrmnvd

2 years

@EbitdaxA @UrInsideMan @AlexNoonan6 where do you work so we can all avoid you

0

1

128

pedram

@pdrmnvd

2 years

@imightbemary Learning to push back is one of the most underrated skills for a DS/analyst. I’m constantly asking: why do we need this? what are we trying to solve? how will you use this data to decide? what does good look like? 80% of my job is cleaning the data the other 20% is being annoying

4

12

135

pedram

@pdrmnvd

1 year

Hey friends. What's your workflow for data analysis? Back when I used to have a real job, I lived in R Studio's world as much as possible. Querying against a DWH, ad-hoc analysis, and creating notebooks to share. But that was years ago what's your workflow looks like today?

42

11

134