The Aggregate Weekly NewsletterđŹOctober 7, 2019
ASR Edition
Hello! I am Lars E. Schonander, a writer for MediaFile and a blogger on international affairs, tech, and general wonkery. Happy Monday! Here is my weekly newsletter with a weekly analysis with interesting data, along with links related to things I found particularly interesting that week. Any Questions? Send me a message or just respond to this email!
The Weekly Data:
Last weekend, I attended hackNY, a hackathon hosted by the non-profit hackNY. For the project our team worked on, we decided to do a social impact-focused project. For this project, I worked on the backend, tinkering with various machine learning algorithms to find the one with the best predictive power, but was also not a black-box model.
The data we used in this came from the 2016 Annual Survey of Refugees, a survey primarily run by the Urban Institute every few years to look at life outcomes. The data was originally meant for SAS, but R can open those files. Interestingly, the factors are labeled, meaning they can be converted into a textual or numeric representation, which is important, because some data is continuous and should be made numeric, and sometimes itâs discrete, and should be converted to its labels. As with many surveys, the data also needed to be cleaned, as for many of the continuous columns 99 represented a refused response. Cleaning these numbers is important to provide more accurate data to a machine learning algorithm along with having a more accurate dataset for exploratory analysis.
A brief lesson on human capital. In most situations, there is a high relation between years of schooling and earnings, as schooling provides the necessary human capital to work in certain types of jobs. Unfortunately, this human capital is devalued if one immigrated to a new country, as people do not value foreign degrees as much as native ones. This is highlighted below.
While there are some outliers (which will be explained later) for the most part, there is no major increase in earnings for refugees even if they many years of education.
One can look into the details of income per hour by degree, and the mean around 11.75, and the fact one standard deviation out is still only 15 dollars per hour shows how much refugees lose in social status when they come to the United States, because of how little they get paid. A classic example of this issue being, an engineer from India or Venezuela, that has to work as a janitor because companies in the United States donât accept the workerâs degree or certifications.
The exception, however, is, those with medical degrees.
For the model, I used a simple linear model, mainly for the ease of deploying to scikit-learn. However, it did have a respectable R^2 and looking at the estimate for the variables tells a slightly different story then the one told above.
Income_Per_Hour ~ Years_of_Schooling + English_Skill_Now + Degree_Type + Industry
Income_Per_Hour: Continious
Years_of_Schooling: Continious
English_Skill_Now: Discrete
Degree Type: Discrete
Industry: Discrete
The main group that does well in the United States regarding the previous industry are engineers, which I am unsure why, but is fascinating. The most important factor in this regression was the ability to speak English well.
As an additional note, before cleaning the Income_Per_Hour variable, the intercept was at 25, which was inaccurate. It took getting rid of the numbers that were being used to note different responses to get the intercept to become more accurate. This also led to a higher R^2.
While P-Values, which repersent statistical significace are important, the R^2 is important because it repersents how much variance is dependent on the independent variables used. Having a high R^2 means that the dependent variable depends very much on the indepdent variables chosen.
Now, some linksâŠ
Rob J Hyndman: Tidy Forecasting in R
The fable package for doing tidy forecasting in R is now on CRAN. Like tsibble and feasts, it is also part of the tidyverts family of packages for analysing, modelling and forecasting many related time series (stored as tsibbles).
For a brief introduction to tsibbles, see this post from last month.
Here we will forecast Australian tourism data by state/region and purpose. This data is stored in theÂ
tourism tsibble whereÂTrips contains domestic visitor nights in thousands.
You Shu: Jiang Shigongâs empire strikes back
There is an influential school of thought in Beijing which believes that the world would be way better off if it was run from Beijing rather than Washington. In a recent essay, Professor Jiang ShigongïŒćŒșäžćïŒoutlines a new theory of empire â and though heâs not explicit about his claim, he clearly believes that Americaâs global leadership is doomed, and that China should work hard to replace it.
Jiang Shigong: Jiang Shigong on âPhilosophy and History: Interpreting the âXi Jinping Eraâ through Xiâs Report to the Nineteenth National Congress of the CCPâ
This essay by Jiang Shigong ćŒșäžć (b. 1967), published in the Guangzhou journal Open Times (ćŒæŸæ¶ä»Ł) in January 2018, aims to be an authoritative statement of the new political orthodoxy under Xi Jinping äč èżćčł(b. 1953) as Xi begins his second term as Chinaâs supreme leader. It offers a new reading of modern Chinese history in general and the history of the Chinese Communist Party (CCP) in particular, arguing that Xi Jinpingâs âthoughtâ (sixiang ææł) is the culmination of a centuryâs historical process and philosophical refinement, produced through the ongoing dialectic of theory and practice. This is âXi Jinping Thought on Socialism with Chinese Characteristics for a New Eraâ äč èżćčłæ°æ¶ä»ŁäžćœçčèČ瀟äŒäž»äčææł, which Jiang defends as the new ideological superstructure to the material base of Chinaâs economy after nearly forty years of âsocialism with Chinese characteristicsâ.
Joshua Brustein (Bloomberg Businessweek): Techâs Most Controversial Startup Now Makes Drone-Killing Robots
Founded by Palmer Luckey and backed by Peter Thiel, Anduril is rekindling the connection between the American military and Silicon Valley.
Juro Osawa (The Information): Travis Kalanick Looks to China for CloudKitchens Expansion
Travis Kalanick is trying again in China. The former Uber CEOâs latest startup, kitchen-rental firm CloudKitchens, has quietly acquired six Chinese startups this year as he looks to expand the business into a global leader.
Earlier this year, Kalanick bought a Shanghai-based startup called Jike Alliance, one of the leading players in Chinaâs kitchen-rental market, based on its number of tenants and revenue, according to the startupâs co-founder. Kalanickâs team also has bought five other kitchen and property-related companies in Beijing, Hangzhou and Chengdu, according to Chinaâs corporate registries and people familiar with the matter. It could not be learned how much Kalanick paid for the companies.
What Iâm Reading
I am currently working my way through Eric Hofferâs book The True Believer. Something that Eric notes that is often ignored is that revolutionary movements are often not led by the lower-class, they are in practice led by disgruntled elites or members of the middle-class. For example, many of the Bolsheviks were dissident writers. The American Revolution was led by disgruntled political and merchant elites who felt like the United Kingdom was not representing them properly. While the down and out create an excellent base for a mass movement, they are often not the leaders.
What Iâm Working On
Still writing my thesis, I had to update to V4 of the GitHub API, which uses GraphQL. Like V3, it is also a very nested API, but it requires slightly different methods to grab the nested data.
I also have a super day with a fintech consulting firm in a few weeks, so I am nervous and excited about that.
Thanks!
Thanks for taking the time to read this, I will be back next Monday. In the meantime, you can follow me on Twitter or reach out via email.