Mininook

Musings on Christianity, Politics, and Computer Science Geekery

Page 3 of 4

Christin: Seeking a fix: Measuring, analyzing and disrupting unlicensed online drug sales

Interesting points from the talk

  • Drugs in different countries have different names, so they had to do matching
  • Use the Jacard distance to find related pharmacies

Interesting points to look into for research:

  • spinglass clustering algorithm
  • visualizations for spinglass

https://www.andrew.cmu.edu/user/nicolasc/

Gerber: Natural Language Processing for Predictive Technology

SIE Colloquium by Matthew Gerber, Research Assistant Professor in the Systems and Information Engineering Department.

The PTL group has 2 faculty, 10 grad students, and collaborators at the health system.

Predicting crime using twitter:

  • Conventional warfare had easily identified forces and open conflict with direct attacks (friends/enemies). The US has no conventional military peers. The US us dealing with asymmetric warfare (asymmetry in size, power, funding, influence). Our enemies have tactical advantages.
  • Monitoring via hot-spot maps
    1. Problems: very specific to the are you're studying and it's retrospective. Can't take yesterday's model and predict on a different place today.
  • Overview of the approach
    1. Gather information on potential crime correlates (Incident Layer, Grid Layer, Demographic Layer, Spatial Layer). Ex: newar military outpost? religious site? Income levels and ethnic tension, and prior history (each on a different layer). Want to take these information and create a statistical model.
    2. Text provides a problem: unstructured text abounds. These short tweets should be helpful: "The second blast was caused by a motorcycle bomb targeting a minibus in the Domeez area in the south of the city. That needs to be read by a human or automated approach (this talk).
    3. Automatically integrate unstructured text: add some new layers from the previous model (Twitter Layer, Newswire Layer, ...).
  • He's looking at tweets from the Chicago area (collecting in the basement of olsson--time, text, etc). A few topics: 1) flight(0.54), plane(0.2), terminal(0.11),... ; 2) shopping (0.39), buy(--),...
    1. Mapping these n topics to heat map of Chicago. Can see where certain things are being talked about.
    2. Unsupervised topic modeling
      • Latent Dirichlet allocation (Blei et al 2003)
      • A generative story (2 topics). Outside of these documents live topics. We can generate these. Do a similar thing with the documents (grab a dirichlet distribution and produce another--a distribution of topics that the document consists of). Want to pick a topic from that distribution to generate a word. (generate by repeating this process).
      • Gather tweets from a neighborhood, tokenize and filter words, identify topic probabilities by LDA, compute probability of crime P(Crime) = F(0.15,0.74,...,f_n). The question what is f?
        1. \frac{1}{1+e^{-\left(\beta_0 + \prod_{b=1}^n \beta_bf_b(p)\right)}}.
        2. Find the beta coefficients that give the best function
      • Training
        • Establish training window (1/1/13-1/31/13)
        • Lay down non-crime points
        • lay down crime points from training window
        • Compute topic neighborhoods
        • compile training data (use Kernel Density Estimate (?) that adds historical data to the model)
      • Evaluation
        • Want to find the smallest place boundaries with the highest crime levels.
        • Do people actually talk about crime on twitter? (that's the big question-- but gangs do trash-talk about their crimes, etc)
        • Baseline for comparison was the kernel density estimation (based on past, where is crime likely to occur?)
        • They do well with twitter data model + KDE over just KDE for certain results: prostitution, battery.
        • They are worse with other topics/crime: homicide, liquor law violations.
        • AUC improvement for 22 of 25 crime types, with average peak improvement of 11 points
  • Clinical Practice Guidelines
    • Want to formalize using natural language processing
    • Sentences have a specific order: they're using NLP and parsing English sentences. (concern: context sensitivity of English)
    • Want to annotate the text with semantic labels (not XML, though).
    • Precisions: temporal identifiers 28% are identified; others average around 50%, with the top around 75-80%
    • Warning: need to make sure that fully automated isn't used alone, as there could be things that automated analysis would miss that could be life-threatening.
  • The big picture
    • Want to get structured information from unstructured text data through Natural Language Processing

Command Line Master

Wanted to post the craziest command line script I've used in a long time.  Used to convert names listed in XML tags in an EAC-CPF record to filenames to copy.

grep -h -o -P "<relationEntry>(.*?)</relationEntry>" *.xml
 | sed -e 's/<[a-zA-Z0-9\/\+]*>//g'
 | awk '{print tolower($0)}'
 | sed -e 's/[ ,.:]\+/\-/g'
 | sed -e 's/$/cr.xml/g'
 | while read x ; do cp /data/production/data/$x eac_data/. ; done

Questions

Go therefore and make disciples of all the nations, baptizing them in the name of the Father and the Son and the Holy Spirit, teaching them to observe all that I commanded you; and lo, I am with you always, even to the end of the age.

Matthew 28:19-20 (NASB)

We have some amazing youth at First Baptist Richmond, I must say.  Bart asked a few questions this evening as our time was ending.  The first I had planned for, since we had spoken about it a few minutes before.  Initially, he asked me "if you could ask God for anything, and knew He would give it to you, what would you ask for?"  That question was easy, or so I thought.  I know some would have jumped at the chance to ask for something or someone in this life, or perhaps tomorrow's winning lottery numbers in jest, but I knew I would want to ask neither of those.  I'd ask for the plans: those things that would both bring me the greatest joy and give God the greatest glory.  A life in which I'm not constantly discerning God's will for my life, but knowing 1) that I know what I'm supposed to do and 2) that I'd be already there.

After everyone arrived and settled, Bart dropped one word that completely changed the intent and depth of the question:

If you could directly ask God anything, and you knew He would answer you, what would you ask?

So, what would you ask?  What would I ask?  Remember back to Genesis, where God is walking in the garden with Adam and Eve; face-to-face discussions with God.  One student asked the question that just amazed me.  She said that she'd ask how God began, if He could explain it in terms we could understand.  We've been taught that God always existed and will exist, and that He is outside our time, but what's His story?  Can you imagine that kind of chat, sitting in rocking chairs on the back porch of God's house, staring into the sunset, learning about Him?

The second, however, took me by surprise, and it immediately brought a lot to mind even though it seemed nearly impossible to answer.

If God were to directly ask you one question right now, what would He ask of you?

My first thought was, what have you been doing with the time I gave you? Or, why don't you trust Me? Or... The list goes on.  Now, looking back at the first question and answer, I'm starting to think I may already know what that plan is; or at least the direction.  Jesus' call in Matthew 28 is a homing beacon to both questions.  Over the next few months, I need to refocus on Jesus' call to us as Christians, knowing that I want to find the place that best serves Him and gives joy.  It isn't easy, and hasn't been over the past few years, but I know there's a place that my abilities and talents will directly serve to bring the Kingdom of God to earth.

Romans 2:12-29

The original plan was to break up the text a bit more, but the last few weeks have been about how the world is sinful, and there is still another week of that to go before we get to the hope that is to come. So, let's consider the rest of Romans 2 this week.

For all who have sinned without the Law will also perish without the Law, and all who have sinned under the Law will be judged by the Law; for it is not the hearers of the Law who are just before God, but the doers of the Law will be justified.

This first sentence brings up a lot of questions. Specifically, it states that those who are just before God are not decided based on whether they have heard the Law, but by whether they have done the Law. Paul is saying here that you don't have to have even heard the Law (here we are talking about the Old Testament's law) to be justified. You just have to do it. How? It is about the attitude of the heart, as Paul continues describing what the Gentiles are doing:

For when Gentiles who do not have the Law do instinctively the things of the Law, these, not having the Law, are a law to themselves, in that they show the work of the Law written in their hearts, their conscience bearing witness and their thoughts alternately accusing or else defending them, on the day when, according to my gospel, God will judge the secrets of men through Christ Jesus.

In the next section Paul is talking to the Jews who are relying on the Law for salvation. Rather than look at it critically, I'd like to look at how it relates to Christianity today.

But if you [...{1}] rely upon the Law and boast in God, and know His will and approve the things that are essential, being instructed out of the Law, and are confident that you yourself are a guide to the blind, a light to those who are in darkness, a corrector of the foolish, a teacher of the immature, having in the Law the embodiment of knowledge and of the truth, you, therefore, who teach another, do you not teach yourself? You who preach that one shall not steal, do you steal? You who say that one should not commit adultery, do you commit adultery? You who abhor idols, do you rob temples? You who boast in the Law, through your breaking the Law, do you dishonor God? For "THE NAME OF GOD IS BLASPHEMED AMONG THE GENTILES BECAUSE OF YOU," just as it is written.

Paul asks those who rely on the Law to realize that they might be breaking it. And also, because the Gentiles see the hypocritical nature of this Law following that it turns them off from God. How often does that happen today? How often do non-Christians see the church today and get utterly turned off from God? We as Christians know the rest of the story: "For by grace you have been saved through faith; and that not of yourselves, it is the gift of God; not as a result of works, so that no one may boast" (Ephesians 2:8, 9 NASB). So many churches, especially here in America, come together as collections of saints and forget that we are all sinners; we screw up on a daily basis. We gather on Sunday mornings ignoring our sins, not asking the hard questions and getting to the root of how each other is really doing. We give off this glow that our lives are perfect and sin-free, but we know they are not. How does that look to a non-Christian? Unattainable? How about hypocritical?

Full Text

For all who have sinned without the Law will also perish without the Law, and all who have sinned under the Law will be judged by the Law; for it is not the hearers of the Law who are just before God, but the doers of the Law will be justified. For when Gentiles who do not have the Law do instinctively the things of the Law, these, not having the Law, are a law to themselves, in that they show the work of the Law written in their hearts, their conscience bearing witness and their thoughts alternately accusing or else defending them, on the day when, according to my gospel, God will judge the secrets of men through Christ Jesus. But if you bear the name "Jew" and rely upon the Law and boast in God, and know His will and approve the things that are essential, being instructed out of the Law, and are confident that you yourself are a guide to the blind, a light to those who are in darkness, a corrector of the foolish, a teacher of the immature, having in the Law the embodiment of knowledge and of the truth, you, therefore, who teach another, do you not teach yourself? You who preach that one shall not steal, do you steal? You who say that one should not commit adultery, do you commit adultery? You who abhor idols, do you rob temples? You who boast in the Law, through your breaking the Law, do you dishonor God? For "THE NAME OF GOD IS BLASPHEMED AMONG THE GENTILES BECAUSE OF YOU," just as it is written. For indeed circumcision is of value if you practice the Law; but if you are a transgressor of the Law, your circumcision has become uncircumcision. So if the uncircumcised man keeps the requirements of the Law, will not his uncircumcision be regarded as circumcision? And he who is physically uncircumcised, if he keeps the Law, will he not judge you who though having the letter of the Law and circumcision are a transgressor of the Law? For he is not a Jew who is one outwardly, nor is circumcision that which is outward in the flesh. But he is a Jew who is one inwardly; and circumcision is that which is of the heart, by the Spirit, not by the letter; and his praise is not from men, but from God. (Romans 2:12-29 NASB)

Notes

  1. I removed a small portion of this verse, where Paul says "but if you bear the name 'Jew'," in which he is talking to the Jews in Rome. However, I think it also has a strong application to Christianity today.
« Older posts Newer posts »

© 2018 Mininook

Theme by Anders NorenUp ↑