Juice, Errr, Ohh…..

Peter Linneman’s Real Estate 209 message

One of the most memorable lessons I took from college was something my professor Peter Linneman said on the first day of his class to a room full of eager, self-confident, and somewhat arrogant Wharton undergraduates back in September of 2004.

His point was simple yet powerful.

Those of us “coastal elites” who obsess over the day-to-day come-up-ins of something like financial world, don’t forget, most of the rest of the people in the United States don’t care and it isn’t because they are stupid it is because whatever it is probably isn’t that important.

If you set out to build to service the needs of the masses don’t just blindly assume they care about all the things you and your network care about.

Peter’s message was, and still is an important one. Don’t take yourself too seriously and don’t think that your fancy education makes you qualified to know what is best for everyone or what will be successful.

Over the years Peter has grown to be one of my most respected mentors and overall favorite people. I often reflect on his fantastic Life Lessons and it all started with that lesson from his first day of class.

Peter’s expertise is in real estate finance and that message was in the context of our class but I found what he had to say has major power in the world of venture investing. In fact, I have found some of the most extreme cliquish mentality among some in the most prominent and powerful roles in Silicon Valley.

When investing in an venture funded idea, be it a physical product like Juicero or a software platform and one of my favorite seed personal investments OpenGov there are 2 key variables to think about in terms of the market for the idea.

• Scalability
• Adoption

Is the product or technology something that appeals to people not just in Los Angeles or Brooklyn but in Cambria Indiana or Waterbury Connecticut?

Do enough people in this country or world REALLY need whatever the pitch is to compensate me for my risk? One needs to really think about these questions as they diligence a pitch.

I learned this lesson the hard-way on some of my earliest investments where I didn’t think long and hard enough about those questions and funded ideas that tried to solve “coastal only” problems.

Needless to say those investments didn’t pan out but the taught me a good lesson, something I’d already been taught by one of my favorite teachers, ask yourself how someone very different from you benefits from the idea seeking funding and no matter how much may be convinced the world needs whatever the idea being pitched is, try playing devil’s advocate for a bit and see how you come out.

I am not assuming that investors in Juicero didn’t do these things but my guess is that if some had followed this advice a bit more they would have avoided what will likely be a 100% loss on this investment.

Now lets get in to the meat of this tutorial.

Part 1: Peeling back the Juicero onion

In this section we are going to demonstrate how to use fundManageR to explore Juicero’s finances and entities that may have exposure to their likely bankruptcy.

After that we are going to use gdeltr2 to see what the media has been saying about the recent events at Juicero.

devtools::install_github("abresler/fundManageR")
library(fundManageR)
library(dplyr)  # install.packages('dplyr') if you don't have it
library(highcharter)  # install.packages('highcharter') if you don't have it

Juicero’s verified capital raises

Now we are ready to use fundManageR’s EDGAR wrapper to explore and visualize the data related to Juicero’s verified capital raises.

The function get_data_sec_filer will allow us to explore the data.

df_juicero <- fundManageR::get_data_sec_filer(entity_names = "Juicero", parse_form_d = TRUE)

Once the function is complete you will notice a number of data frames in your environment containing the information parsed from EDGAR.

You can explore the individual data frames if you wish but for now we want to focus on the dataFilerFormDsOfferingSalesAmounts data which contains information about the amount of capital raised in each filed Form D.

You will notice there are 2 entities that matched our search termJuicero.

We want to focus on the Juicero corporate entity whose CIK number is 1588182. The other entity appears to be a sidecar that may be affiliated with a Juicero financing round but we aren’t sure so we should exclude that.

After we select only the Juciero corporate entity we need to tidy the data a bit so we can focus on the amount of capital solicited versus the amount of capital raised. W can then take that and interactively explore the end result.

library(dplyr) # install.packages(dplyr)
library(tidyr) # install.packages(tidyr)
library(highcharter) # install.packages("highcharter")

## Filter and Munge
df_juicero_d <- 
  dataFilerFormDsOfferingSalesAmounts %>% 
  filter(idCIK == 1588182) %>% 
  select(dateFiling, amountSoldTotal, amountOfferingTotal) %>% 
  gather(item, value, -dateFiling) %>% 
  mutate(item = item %>% as.factor())

df_juicero_d %>% 
  hchart(type = "area", hcaes(x = dateFiling, y = value, group = item)) %>% 
  hc_tooltip(sort = TRUE, table = TRUE) %>%
  hc_tooltip(split = TRUE) %>%
  hc_add_theme(hc_theme_smpl()) %>% 
  hc_title(text = "Juicero Inc Form D Verified Capital Raises",
           useHTML = TRUE) %>%
  hc_subtitle(
    text = "Data from the SEC via fundManageR",
    useHTML = TRUE
  ) %>% 
  hc_plotOptions(series = list(
    marker = list(enabled = FALSE),
    cursor = "pointer",
    states = list(hover = list(enabled = FALSE)),
    stickyTracking = TRUE
  )) %>% 
  hc_xAxis(
    type = "datetime",
    gridLineWidth = 0,
    lineWidth = 1,
    lineColor = "black",
    tickWidth = 1,
    tickAmount = 15,
    tickColor = "black",
    title = list(text = "Filing Date", style = list(color = "black")),
    labels = list(style = list(color = "black"))
  )

What can we take from this?

There is no way for us tto confirm that Juicero actually raised $120,000,000.

In fact we can only be sure that they raised $19,881,830.

That said, absence of evidence does not mean definite absence. There are forms of capital that often go undisclosed to the SEC and it is possible that Juicero raised the amount of money we cannot account for via those means. That said, I find it highly unlikely that Juicero really raised $120,000,000 and if I were an editor at a media entity I would not use that information based upon the facts that we have at this point.

It turns out we maybe able to fill in the missing alleged money another way also using fundManageR and that is by searching through the text of the last 3 years of every EDGAR filing.

df_juicero_ft <- fundManageR::get_data_edgar_ft_terms(search_terms = "\"Juicero\"", 
    include_counts = FALSE)

library(DT) # install.packages("formattable")
library(glue) # install.packages("glue")

## 
## Attaching package: 'glue'

## The following object is masked from 'package:dplyr':
## 
##     collapse

df_juicero_ft %>% 
  select(dateFiling:urlSECFiling) %>% 
  select(-countFilings) %>% 
  mutate(urlSECFiling = glue::glue("<a href = '{urlSECFiling}' target = '_blank'>Filing</a>")) %>% 
  group_by(idCIKFiler, dateFiling) %>% 
  arrange(desc(dateFiling)) %>% 
  datatable(filter = 'top', options = list(
  pageLength = 5, autoWidth = TRUE
), escape = FALSE)

What does this tell us?

Our full text search unearthed some valuable information:

• Campbell’s Soup invested in $13,500,000 in Juicero and a high level employee left to become CEO of Juicero
• Juicero licensed important technology from another public company, Landec Corporation
• A commercial mortgage was securitized with a Juicero rent comparable
• Juicero’s equity was placed in a couple of sidecars including at Next Play Capital and A-List ventures, an Angelist syndicate.

None of this information confirms Juicero has actually raised $120,000,000.

The new Campbell Soup disclosure and the sidecars formed in 2016 do lead me to believe the amount of funding Juicero raised is likely higher than the confirmed amount from the Form Ds but we cannot be absolutely sure.

Going these functions may be useful assisting us in monitoring any new developments as Juicero is dissolved.

It is possible that upon an actual bankruptcy, Campbell’s Soup will be forced to file information with the SEC that details what happened to their investment but we will have to just wait and see.

GDELT, what happend at Juicero?

It may be some time until we get an accurate depiction of what caused the demise of Juicero but it is still worthwhile to sift through the entire media universe to find stories about Juicero for us to explore.

To do this we will use the gdeltr2 package. For those of you new to gdeltr2 I recommend you take a look at my last blog post which walks you through an in depth tutorial of the package and the functions we are about to use to explore this topic.

library(gdeltr2)  # devtools::install_githu('abresler/gdeltr2') if you need the package

dates <- gdeltr2::generate_dates(start_date = Sys.Date() - 84, end_date = Sys.Date(), 
    time_interval = "days")

gdeltr2::get_data_ft_v2_api(terms = c("\"Juicero\""), images_ocr = c("Juicero"), 
    dates = dates, timespans = NA)

Timeline

Next we will generate an interactive showcasing the volume of stories matching our 2 Juicero search parameters. This will help us identify when the media picked up on things going bad at the company.

gdeltr2::get_data_ft_v2_api(terms = c("\"Juicero\""), images_ocr = c("Juicero"), 
    modes = "timelinevolinfo", timespans = "12 Weeks")

Wordcloud

Finally we will generate wordclouds of the parsed text over a 2 and 12 week time span to allow us to quickly explore the words most associated with the Juicero stories.

gdeltr2::get_data_ft_v2_api(terms = c("\"Juicero\""), images_ocr = c("Juicero"), 
    modes = c("WordCloudEnglish"), trelliscope_parameters = list(rows = 1, columns = 1), 
    timespans = c("12 weeks", "2 weeks"))

Wrapping up Part 1

This wraps up our direct exploration of Juicero. I hope it demonstrated how to use each of my packages to unearth meaningful information about an entity and that it showcased how easily you can process large volumes of information to try to get at potential truths.

In the next section will transition into something completely different but truly the most fun and exciting thing I have done in my programming adventures and that is using machine learning to generate fake start-ups.

Step 0 – Pre-requisites

In addition to R you need Python 2 installed along with the Tensorflow, Keras and Markovify modules.

You can follow the Keras/Tensorflow installation instructions here

To install Markovify you can run this code inside of R

system("pip install markovify")

Once this is completed we get into turning these ideas into code.

Step 1 – Load the packages

Now we are ready to load the packages needed to build, run and explore our functions.

It is important that you install the packages I am using if you don’t already have them.

package_to_load <- c("fundManageR", "purrr", "stringr", "dplyr", "rlang", "reticulate", 
    "keras", "tokenizers", "tidyr", "glue", "DT", "formattable")
lapply(package_to_load, library, character.only = T)

Step 2 – Refine the YC data

Before we get started on the machine learning lets spend a few moments exploring and further refining the YC data.

To acquire YC alumni we use fundManageR:::get_data_ycombinator_alumni function.

Lets bring in this data.

df_yc <- fundManageR::get_data_ycombinator_alumni()

Let’s explore the structure of this data to see if we can come up with some additional filters.

df_yc %>% 
  glimpse()

## Observations: 1,320
## Variables: 9
## $ nameCompany        <chr> "COCUSOCIAL", "WILDFIRE", "HOTELFLEX", "ONC...
## $ isCompanyDead      <lgl> FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, F...
## $ isCompanyAcquired  <lgl> FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, N...
## $ batchYC            <chr> "s2017", "s2017", "s2017", "s2017", "s2017"...
## $ yearYC             <dbl> 2017, 2017, 2017, 2017, 2017, 2017, 2017, 2...
## $ nameSeasonYC       <chr> "SUMMER", "SUMMER", "SUMMER", "SUMMER", "SU...
## $ urlCompany         <chr> "http://www.cocusocial.com", "https://www.w...
## $ verticalCompany    <chr> "Marketplace", NA, "B2B", NA, "Education", ...
## $ descriptionCompany <chr> "Marketplace for local activities.", "Wildf...

Thinking a step ahead towards the machine learning we are about to do it should be useful to really refine the YC data to allow us to generate machine learning output for specific company verticals, seasons, years, and so on.

To do this we will build a function that refines the YC alumni data for the parameters mentioned above in addition to a few other parameters and data cleaning tasks.

The function takes a data frame containing the YC alumni and returns the filtered results you specify.

filter_yc_alumni <-
  function(data,
           years = NULL,
           verticals = NULL,
           season = NULL,
           exclude_acquisitions = TRUE,
           only_defunct_companies = FALSE) {
    ## remove acronyms
    data <- 
      data %>%
      separate(nameCompany, sep = '\\(|\\/', c("nameCompany", "acronymCompany")) %>%
      mutate(acronymCompany = acronymCompany %>% str_replace_all('\\)', '')) %>%
      mutate_if(is.character,
      str_trim)
    ## make descriptions all upper case
    data_ycombinator <-
      data %>%
      filter(!is.na(descriptionCompany)) %>%
      mutate_at(.vars = c("descriptionCompany"),
                .funs = str_to_upper)
    
    ## Clean the descriptions
    data_ycombinator <-
      1:nrow(data_ycombinator) %>%
      map_df(function(x) {
        df_row <- 
          data_ycombinator %>% slice(x)
        
        company <-
          df_row$nameCompany
        
        remove_text <-
          glue::glue("{company} IS")
        
        df_row %>%
          mutate(
            descriptionCompany = 
              descriptionCompany %>% 
              str_replace_all(remove_text, "") %>% 
              str_replace_all(company, '') %>%
              str_trim()
          )
      })
    
    # Filter out acquisitions
    if (exclude_acquisitions) {
      data_ycombinator <-
        data_ycombinator %>%
        filter(!isCompanyAcquired)
    }
    ## Return only dead companies
    if (only_defunct_companies) {
      data_ycombinator <-
        data_ycombinator %>%
        filter(isCompanyDead)
    }
    ## Specify verticals
    if (!purrr::is_null(verticals)) {
      glue::glue(
        "Limiting training YC data to {length(verticals)} verticles:\n
        {str_c(verticals, collapse =', ')}
        "
      ) %>% 
        message()
      
      data_ycombinator <-
        data_ycombinator %>%
        filter(verticalCompany %in% verticals)
    }
    
    if (!purrr::is_null(years)) {
      glue::glue(
        "Limiting training YC data to the following years:\n
        {str_c(years, collapse =', ')}
        "
      ) %>%
        message()
      
      data_ycombinator <-
        data_ycombinator %>%
        filter(yearYC %in% years)
    }
    
    if (!purrr::is_null(season)) {
      glue::glue(
        "Limiting training YC data to the following season:\n
        {str_c(season, collapse =', ')}
        "
      ) %>% 
        message()
      
      data_ycombinator <-
        data_ycombinator %>%
        filter(nameSeason %in% season)
    }
    data_ycombinator
  }

Step 3 – Build text generating neural network

Now we are ready to build a function that uses text to generate text via a neural network.

I’ve coined this function kerasify_text.

I am not getting into the specifics of how neural networks work. For those who are curious and have a bit of time I recommend this video.

What we are going to do is build function that from a vector of text, mixed with a host of neural network parameters spits out machine learning driven text.

These parameters are the number of epochs, diversity, batch size, learning rate, maximum number of characters and maximum sentence length.

We are also going to provide a label for what the text output we are generating should be called and whether we want to see the results as the neural network works its magic.

The final result will be a data frame with the text and information about how the machine learning output was generated.

kerasify_text <-
  function(input_text = text_blob,
           output_column_name = "nameCompanyFake",
           regurgitate_output = TRUE,
           epochs = 5,
           learning_rate = 0.01,
           maximum_length = 3,
           maximum_iterations = 50,
           batch_size = 128,
           diversities =  c(0.2, 0.5, 1, 1.2),
           maximum_chars = 500) {
    
    if (!'input_text' %>% exists()) {
      stop("Please enter input text")
    }
    
    ### MODIFIED FROM https://keras.rstudio.com/articles/examples/lstm_text_generation.html
    iterations <- 1:maximum_iterations
    max_chars <- 1:maximum_chars
    
    input_text <-
      input_text %>%
      str_c(collapse = "\n") %>%
      tokenize_characters(strip_non_alphanum = FALSE, simplify = TRUE)
    
    chars <-
      input_text %>%
      unique() %>%
      sort()
    
    glue::glue("Total characters for neural network: {length(chars)}") %>%
      message()
    
    # cut the input_text in semi-redundant sequences of maximum_length characters
    dataset <-
      map(seq(1, length(input_text) - maximum_length - 1, by = 3),
          
          ~ list(sentence = input_text[.x:(.x + maximum_length - 1)], next_char = input_text[.x + maximum_length]))
    
    dataset <- transpose(dataset)
    # vectorization
    input_matrix <-
      array(0, dim = c(length(dataset$sentence), maximum_length, length(chars)))
    output_array <-
      array(0, dim = c(length(dataset$sentence), length(chars)))
    
    for (i in 1:length(dataset$sentence)) {
      input_matrix[i, ,] <- sapply(chars, function(x) {
        as.integer(x == dataset$sentence[[i]])
      })
      
      output_array[i,] <- as.integer(chars == dataset$next_char[[i]])
      
    }
    
    ### Model Definition
    
    model <-
      keras_model_sequential()
    
    model %>%
      layer_lstm(batch_size, input_shape = c(maximum_length, length(chars))) %>%
      layer_dense(length(chars)) %>%
      layer_activation("softmax")
    
    optimizer <-
      optimizer_rmsprop(lr = learning_rate)
    
    model %>%
      compile(loss = "categorical_crossentropy",
              optimizer = optimizer)
    
    
    ## Model Training
    
    sample_mod <- function(preds, temperature = 1) {
      preds <- log(preds) / temperature
      exp_preds <- exp(preds)
      preds <- exp_preds / sum(exp(preds))
      
      rmultinom(1, 1, preds) %>%
        as.integer() %>%
        which.max()
    }
    all_data <-
      data_frame()
    
    for (iteration in iterations) {
      glue::glue("iteration: {iteration} \n\n")
      
      model %>%
        fit(input_matrix,
            output_array,
            batch_size = batch_size,
            epochs = epochs)
      
      for (diversity in diversities) {
        glue::glue("diversity: {diversity} \n\n")
        start_index <-
          sample(1:(length(input_text) - maximum_length), size = 1)
        sentence <-
          input_text[start_index:(start_index + maximum_length - 1)]
        generated <- ""
        
        for (i in max_chars) {
          x <- sapply(chars, function(x) {
            as.integer(x == sentence)
          })
          dim(x) <- c(1, dim(x))
          
          preds <- predict(model, x)
          next_index <- sample_mod(preds, diversity)
          next_char <- chars[next_index]
          
          generated <- str_c(generated, next_char, collapse = "")
          sentence <- c(sentence[-1], next_char)
        }
        
        generated <- generated %>% str_to_upper()
        ## Spits out names
        if (regurgitate_output) {
        glue::glue("\n{cat(generated)}\n")
        }
        
        generated_names <-
          str_split(generated, '\n') %>%
          flatten_chr() %>%
          str_to_upper() %>%
          unique() %>% str_trim() %>% {
            .[!. == '']
          }
        ## Assing to DF
        df <- 
          data_frame(UQ(output_column_name) := generated_names)
        
        
        df <-
          df %>%
          mutate(idIteration = iteration,
                 idDiversity = diversity) %>%
          dplyr::select(idIteration, idDiversity, everything())
        
        all_data <-
          all_data %>%
          bind_rows(df)
      }
    }
    ## Return results
    all_data
  }

Step 4 – Build text generating hidden Markov model

Next we are going to build a Markov model that generates text.

There are ways to achieve this solely using R packages but I want to demonstrate how easy it us to mix Python and R in R using the reticulate package.

Furthermore, I find the Python Markovify module far superior to any R package for Markov based text generation.

The function we are about to build will be called markovify_text.

It takes a vector of strings, a Markov state size, and maximum sentence length to produce a data frame containing your labeled output for the the number of times you wish to generate text.

Please note, the higher the markov_state_size parameter the more coherent the text should sound but the less likely it is the model will find a sentence. For the company descriptions you should go no higher than a state size of 4 which will still generate a significant number of NA values and repeat descriptions.

On the flip side, a low state size will generate significantly stranger sounding text with a lower likely hood of NA values and repeated text.

The default state size of 2 produces a happy medium between funny/strange and coherent text output.

markovify_text <-
  function(input_text,
           markov_state_size = 2,
           output_column_name = "descriptionCompanyFake",
           maximum_sentence_length = 1200,
           iterations = 50,
           return_message = TRUE,) {
    ## ensure inputs are integers
    markov_state_size <-
      as.integer(markov_state_size)
    maximum_sentence_length <-
      as.integer(maximum_sentence_length)
    
    ## import markovify from Python
    mk <-
      import("markovify")
    
    ## build the model
    markov_model <-
      mk$Text(input_text = input_text, state_size = markov_state_size)
    
    ## Build the data frame
    df_text <-
      1:iterations %>%
      map_df(function(x) {
        if (return_message) {
          glue::glue('Generating Markov model text: {x} of {count}') %>%
            message()
        }
        
        text_output <-
          markov_model$make_short_sentence(max_chars = maximum_sentence_length)
        ## if no result try again
        if (text_output %>% length() == 0) {
          text_output <-
            markov_model$make_short_sentence(max_chars = maximum_sentence_length)
        }
        # if still no result NA
        if (text_output %>% length() == 0) {
          text_output <- NA
        }
        ## return data frame
        data_frame(idRow = x, UQ(output_column_name) := text_output)
      })
    
    df_text
  }

Unlike the kerasify_text function, markovify_text is quite fast.

As I alluded to above, depending on the number of states selected, the model may produce an NA value if a sentence cannot be generated.

Also, depending on the number of iterations and state selected you may see repeat output. The more iterations and higher state selection, the more likely you are to have this occur.

Step 5 – Put it all together

We now have prepared the ingredients needed to generate our fake start ups.

Lets put them to use.

df_yc <- 
  df_yc %>% 
  filter_yc_alumni(exclude_acquisitions = TRUE, only_defunct_companies  = FALSE)

yc_companies <- 
  df_yc %>% 
  pull(nameCompany)

df_names <-
  kerasify_text(
    input_text =  yc_companies,
    output_column_name = "nameCompanyFake",
    regurgitate_output = TRUE,
    epochs = 25,
    maximum_length = 4,
    maximum_iterations = 50,
    batch_size = 140,
    maximum_chars = 300
  )

df_cos <- 
  df_names %>% 
  select(nameCompanyFake) %>% 
  distinct() %>% 
  mutate(charName = nchar(nameCompanyFake)) %>% 
  filter(charName %>% between(2, 15)) %>% 
  mutate(idRow = 1:n()) %>% 
  dplyr::select(-c(charName))

company_descriptions <-
  df_yc %>%
  pull(descriptionCompany)

df_descriptions <-
  markovify_text(
    input_text = company_descriptions,
    output_column_name = 'descriptionCompanyFake',
    markov_state_size = 2,
    maximum_sentence_length = 400,
    iterations = nrow(df_cos)
  )

all_data <- 
  df_cos %>% 
  left_join(df_descriptions) %>% 
  dplyr::select(-idRow) %>% 
  filter(!descriptionCompanyFake %>% is.na())

Time to Explore

Now after all this hard work it is time to enjoy the fruits, or more appropriately, the cold-pressed juice pods of our labor 😃😎😎

We generated a few thousand companies, lets take a look at 10 random ones.

To do that we will use the `dplyr::sample_n`` function.

all_data %>% 
  sample_n(10) %>% 
  formattable::formattable()

Take a look at that, 10 machine learning generated start-up ideas that could be the next billion dollar unicorn and probably are a better idea to me at least than Juicero’s pitch.

Now lets take a look at all 4507 machine learning generated start up ideas and make them available to the whole world for the rest of time.

Feel free to peruse, export and even use them for possible start-up ideas of your own. I just ask that you remember me if anything ever comes of the idea and drop me a line about how you encountered this post after you raise your first successful round of funding!

library(DT)
all_data %>%
  datatable(
    filter = 'top',
    extensions = c('Responsive', 'Buttons'),
    options = list(
      dom = 'Bfrtip',
      buttons = c('csv', 'excel', 'pdf', 'print'),
      pageLength = 5,
      autoWidth = TRUE,
      lengthMenu = c(5, 25, 50, 100, 500)
    )
  )

That wraps up the second portion of this post. I encourage you to play around a bit with these functions. Change around the parameters and try filtering the input data to focus on specific verticals or company outcomes and see how the results differ.

Parting notes

First, I want to thank Antonio for inspiring this post.

Your humor and wit lit the fire I needed to finally publicly showcase how to use my packages together and most importantly take on this advanced machine learning task.

I also want to thank the RStudio team.

Their keras and reticulate packages are absolutely incredible and make it seamless to both integrate Python with R. These packages handle all the machine learning heavy lifting with ease and grace

Finally I want to thank everyone reading this.

I hope you found it engaging and useful. There aren’t many tutorials on integrating Python with R or generating text from machine learning functions. For those that were able to complete the second part I hope that I did an adequate job of trying to fill that void in an insightful and engaging manner.

I also encourage you to use these functions with other text corupa that interest you.

You can impress your friends with neural network generated rap music or a markovifyed version of Stephen A. Smith’s twitter.

Use your imagination and machine learning can take you to some fun and interesting places.

Until next time…