Mininook

Musings on Christianity, Politics, and Computer Science Geekery

Page 3 of 4

Gerber: Natural Language Processing for Predictive Technology

SIE Colloquium by Matthew Gerber, Research Assistant Professor in the Systems and Information Engineering Department.

The PTL group has 2 faculty, 10 grad students, and collaborators at the health system.

Predicting crime using twitter:

  • Conventional warfare had easily identified forces and open conflict with direct attacks (friends/enemies). The US has no conventional military peers. The US us dealing with asymmetric warfare (asymmetry in size, power, funding, influence). Our enemies have tactical advantages.
  • Monitoring via hot-spot maps
    1. Problems: very specific to the are you're studying and it's retrospective. Can't take yesterday's model and predict on a different place today.
  • Overview of the approach
    1. Gather information on potential crime correlates (Incident Layer, Grid Layer, Demographic Layer, Spatial Layer). Ex: newar military outpost? religious site? Income levels and ethnic tension, and prior history (each on a different layer). Want to take these information and create a statistical model.
    2. Text provides a problem: unstructured text abounds. These short tweets should be helpful: "The second blast was caused by a motorcycle bomb targeting a minibus in the Domeez area in the south of the city. That needs to be read by a human or automated approach (this talk).
    3. Automatically integrate unstructured text: add some new layers from the previous model (Twitter Layer, Newswire Layer, ...).
  • He's looking at tweets from the Chicago area (collecting in the basement of olsson--time, text, etc). A few topics: 1) flight(0.54), plane(0.2), terminal(0.11),... ; 2) shopping (0.39), buy(--),...
    1. Mapping these n topics to heat map of Chicago. Can see where certain things are being talked about.
    2. Unsupervised topic modeling
      • Latent Dirichlet allocation (Blei et al 2003)
      • A generative story (2 topics). Outside of these documents live topics. We can generate these. Do a similar thing with the documents (grab a dirichlet distribution and produce another--a distribution of topics that the document consists of). Want to pick a topic from that distribution to generate a word. (generate by repeating this process).
      • Gather tweets from a neighborhood, tokenize and filter words, identify topic probabilities by LDA, compute probability of crime P(Crime) = F(0.15,0.74,...,f_n). The question what is f?
        1. \frac{1}{1+e^{-\left(\beta_0 + \prod_{b=1}^n \beta_bf_b(p)\right)}}.
        2. Find the beta coefficients that give the best function
      • Training
        • Establish training window (1/1/13-1/31/13)
        • Lay down non-crime points
        • lay down crime points from training window
        • Compute topic neighborhoods
        • compile training data (use Kernel Density Estimate (?) that adds historical data to the model)
      • Evaluation
        • Want to find the smallest place boundaries with the highest crime levels.
        • Do people actually talk about crime on twitter? (that's the big question-- but gangs do trash-talk about their crimes, etc)
        • Baseline for comparison was the kernel density estimation (based on past, where is crime likely to occur?)
        • They do well with twitter data model + KDE over just KDE for certain results: prostitution, battery.
        • They are worse with other topics/crime: homicide, liquor law violations.
        • AUC improvement for 22 of 25 crime types, with average peak improvement of 11 points
  • Clinical Practice Guidelines
    • Want to formalize using natural language processing
    • Sentences have a specific order: they're using NLP and parsing English sentences. (concern: context sensitivity of English)
    • Want to annotate the text with semantic labels (not XML, though).
    • Precisions: temporal identifiers 28% are identified; others average around 50%, with the top around 75-80%
    • Warning: need to make sure that fully automated isn't used alone, as there could be things that automated analysis would miss that could be life-threatening.
  • The big picture
    • Want to get structured information from unstructured text data through Natural Language Processing

Command Line Master

Wanted to post the craziest command line script I've used in a long time.  Used to convert names listed in XML tags in an EAC-CPF record to filenames to copy.

grep -h -o -P "<relationEntry>(.*?)</relationEntry>" *.xml
 | sed -e 's/<[a-zA-Z0-9\/\+]*>//g'
 | awk '{print tolower($0)}'
 | sed -e 's/[ ,.:]\+/\-/g'
 | sed -e 's/$/cr.xml/g'
 | while read x ; do cp /data/production/data/$x eac_data/. ; done

Questions

Go therefore and make disciples of all the nations, baptizing them in the name of the Father and the Son and the Holy Spirit, teaching them to observe all that I commanded you; and lo, I am with you always, even to the end of the age.

Matthew 28:19-20 (NASB)

We have some amazing youth at First Baptist Richmond, I must say.  Bart asked a few questions this evening as our time was ending.  The first I had planned for, since we had spoken about it a few minutes before.  Initially, he asked me "if you could ask God for anything, and knew He would give it to you, what would you ask for?"  That question was easy, or so I thought.  I know some would have jumped at the chance to ask for something or someone in this life, or perhaps tomorrow's winning lottery numbers in jest, but I knew I would want to ask neither of those.  I'd ask for the plans: those things that would both bring me the greatest joy and give God the greatest glory.  A life in which I'm not constantly discerning God's will for my life, but knowing 1) that I know what I'm supposed to do and 2) that I'd be already there.

After everyone arrived and settled, Bart dropped one word that completely changed the intent and depth of the question:

If you could directly ask God anything, and you knew He would answer you, what would you ask?

So, what would you ask?  What would I ask?  Remember back to Genesis, where God is walking in the garden with Adam and Eve; face-to-face discussions with God.  One student asked the question that just amazed me.  She said that she'd ask how God began, if He could explain it in terms we could understand.  We've been taught that God always existed and will exist, and that He is outside our time, but what's His story?  Can you imagine that kind of chat, sitting in rocking chairs on the back porch of God's house, staring into the sunset, learning about Him?

The second, however, took me by surprise, and it immediately brought a lot to mind even though it seemed nearly impossible to answer.

If God were to directly ask you one question right now, what would He ask of you?

My first thought was, what have you been doing with the time I gave you? Or, why don't you trust Me? Or... The list goes on.  Now, looking back at the first question and answer, I'm starting to think I may already know what that plan is; or at least the direction.  Jesus' call in Matthew 28 is a homing beacon to both questions.  Over the next few months, I need to refocus on Jesus' call to us as Christians, knowing that I want to find the place that best serves Him and gives joy.  It isn't easy, and hasn't been over the past few years, but I know there's a place that my abilities and talents will directly serve to bring the Kingdom of God to earth.

Romans 2:12-29

The original plan was to break up the text a bit more, but the last few weeks have been about how the world is sinful, and there is still another week of that to go before we get to the hope that is to come. So, let's consider the rest of Romans 2 this week.

For all who have sinned without the Law will also perish without the Law, and all who have sinned under the Law will be judged by the Law; for it is not the hearers of the Law who are just before God, but the doers of the Law will be justified.

This first sentence brings up a lot of questions. Specifically, it states that those who are just before God are not decided based on whether they have heard the Law, but by whether they have done the Law. Paul is saying here that you don't have to have even heard the Law (here we are talking about the Old Testament's law) to be justified. You just have to do it. How? It is about the attitude of the heart, as Paul continues describing what the Gentiles are doing:

For when Gentiles who do not have the Law do instinctively the things of the Law, these, not having the Law, are a law to themselves, in that they show the work of the Law written in their hearts, their conscience bearing witness and their thoughts alternately accusing or else defending them, on the day when, according to my gospel, God will judge the secrets of men through Christ Jesus.

In the next section Paul is talking to the Jews who are relying on the Law for salvation. Rather than look at it critically, I'd like to look at how it relates to Christianity today.

But if you [...{1}] rely upon the Law and boast in God, and know His will and approve the things that are essential, being instructed out of the Law, and are confident that you yourself are a guide to the blind, a light to those who are in darkness, a corrector of the foolish, a teacher of the immature, having in the Law the embodiment of knowledge and of the truth, you, therefore, who teach another, do you not teach yourself? You who preach that one shall not steal, do you steal? You who say that one should not commit adultery, do you commit adultery? You who abhor idols, do you rob temples? You who boast in the Law, through your breaking the Law, do you dishonor God? For "THE NAME OF GOD IS BLASPHEMED AMONG THE GENTILES BECAUSE OF YOU," just as it is written.

Paul asks those who rely on the Law to realize that they might be breaking it. And also, because the Gentiles see the hypocritical nature of this Law following that it turns them off from God. How often does that happen today? How often do non-Christians see the church today and get utterly turned off from God? We as Christians know the rest of the story: "For by grace you have been saved through faith; and that not of yourselves, it is the gift of God; not as a result of works, so that no one may boast" (Ephesians 2:8, 9 NASB). So many churches, especially here in America, come together as collections of saints and forget that we are all sinners; we screw up on a daily basis. We gather on Sunday mornings ignoring our sins, not asking the hard questions and getting to the root of how each other is really doing. We give off this glow that our lives are perfect and sin-free, but we know they are not. How does that look to a non-Christian? Unattainable? How about hypocritical?

Full Text

For all who have sinned without the Law will also perish without the Law, and all who have sinned under the Law will be judged by the Law; for it is not the hearers of the Law who are just before God, but the doers of the Law will be justified. For when Gentiles who do not have the Law do instinctively the things of the Law, these, not having the Law, are a law to themselves, in that they show the work of the Law written in their hearts, their conscience bearing witness and their thoughts alternately accusing or else defending them, on the day when, according to my gospel, God will judge the secrets of men through Christ Jesus. But if you bear the name "Jew" and rely upon the Law and boast in God, and know His will and approve the things that are essential, being instructed out of the Law, and are confident that you yourself are a guide to the blind, a light to those who are in darkness, a corrector of the foolish, a teacher of the immature, having in the Law the embodiment of knowledge and of the truth, you, therefore, who teach another, do you not teach yourself? You who preach that one shall not steal, do you steal? You who say that one should not commit adultery, do you commit adultery? You who abhor idols, do you rob temples? You who boast in the Law, through your breaking the Law, do you dishonor God? For "THE NAME OF GOD IS BLASPHEMED AMONG THE GENTILES BECAUSE OF YOU," just as it is written. For indeed circumcision is of value if you practice the Law; but if you are a transgressor of the Law, your circumcision has become uncircumcision. So if the uncircumcised man keeps the requirements of the Law, will not his uncircumcision be regarded as circumcision? And he who is physically uncircumcised, if he keeps the Law, will he not judge you who though having the letter of the Law and circumcision are a transgressor of the Law? For he is not a Jew who is one outwardly, nor is circumcision that which is outward in the flesh. But he is a Jew who is one inwardly; and circumcision is that which is of the heart, by the Spirit, not by the letter; and his praise is not from men, but from God. (Romans 2:12-29 NASB)

Notes

  1. I removed a small portion of this verse, where Paul says "but if you bear the name 'Jew'," in which he is talking to the Jews in Rome. However, I think it also has a strong application to Christianity today.

Romans 2:1-11

Note: sorry for the delay, this week I was recovering from having my wisdom teeth removed.

This week, we're looking at the beginning of the second chapter of Romans. Paul continues to bring the Jews and Christians in Rome onto the same level and to show that they are all sinners in need of God's saving grace.

Therefore you have no excuse, everyone of you who passes judgment, for in that which you judge another, you condemn yourself; for you who judge practice the same things. And we know that the judgment of God rightly falls upon those who practice such things. But do you suppose this, O man, when you pass judgment on those who practice such things and do the same yourself, that you will escape the judgment of God? Or do you think lightly of the riches of His kindness and tolerance and patience, not knowing that the kindness of God leads you to repentance? But because of your stubbornness and unrepentant heart you are storing up wrath for yourself in the day of wrath and revelation of the righteous judgment of God, who WILL RENDER TO EACH PERSON ACCORDING TO HIS DEEDS: to those who by perseverance in doing good seek for glory and honor and immortality, eternal life; but to those who are selfishly ambitious and do not obey the truth, but obey unrighteousness, wrath and indignation. There will be tribulation and distress for every soul of man who does evil, of the Jew first and also of the Greek, but glory and honor and peace to everyone who does good, to the Jew first and also to the Greek. For there is no partiality with God. (Romans 2:1-11 NASB)

First, let us discuss again the faith of the Jewish and Gentile Christians. The Jews were the people of God, by birth. They had both the history and the Law. The Gentiles did not have this inheritance, and therefore their faith looked a bit different. Paul paints a picture of these Jews very similar to what we see of the Pharisees in the gospels: outwards righteous, public followers of the law, but entitled, "unthankful, rebellious, and unrighteous" (3). Luther, in his commentary, sums up this attitude, rephrasing Paul's words to the Jews in Rome as:

"You live a fine outward life in the works of the law, and judge those who do not so live, and know how to teach everyone; you see the splinter in the other’s eye, but of the beam in your own eye you are not aware" (1).

So, what is the punch line? It's simple: don't be hypocritical, we're all sinners, both Jews and Gentiles. We're all sinners, both Jewish Christians and Gentile Christians; no matter where you come from. Anyone who judges others and condemns them for what they've done wrong will not make the judger more righteous. In fact, Paul says that if you judge others, and especially if you commit the same sin, "you condemn yourself."

But all who act thus, of every nation, age, and description, must be reminded that the judgment of God will be according to their real character. The case is so plain, that we may appeal to the sinner's own thoughts. In every wilful sin, there is contempt of the goodness of God. And though the branches of man's disobedience are very various, all spring from the same root. But in true repentance, there must be hatred of former sinfulness, from a change wrought in the state of the mind, which disposes it to choose the good and to refuse the evil. (3)

How do we fix this? As Henry stated in the quote above, it is about the heart of the sinner. Paul calls us to repentance. This is the works vs faith argument, and Paul says here that "God will render to each person according to his deeds." Wait--deeds save? No, not exactly. The beginning of that sentence tells the whole story: it's about the heart. The unrepentant heart stores up for itself wrath and judgment. It doesn't produce good works! Those who by perseverance in doing good (aka, those that have faith and are pursuing God) seek for glory and honor and immortality will be given eternal life, since their faith produces good works. But those who are selfishly ambitious and do not obey the truth, but obey unrighteousness, they will inherit wrath and indignation because of their lack of good-work-producing faith.

This end is for every person, regardless if they grew up in the Law or not, Jew or not, righteous in the eyes of man or not. That is Paul's point in these verses: we are all sinners and cannot fulfill all the commandments and the Law.

How much has Christianity today become like that of the Jews and Jewish Christians that Paul is addressing here? How many of us think we are righteous enough to earn salvation by keeping the law?

Jesus, in his conversation with Simon in Luke 7:36-50, also illustrates our sinfulness. A sinful woman, likely a prostitute, has come into the house of the pharisee to see Jesus. All the "righteous" are in shock as she comes in, weeps over Jesus' feet, washes them with her hair, then anoints them with perfume. When Simon condemns her (to himself), Jesus tells him this parable:

"A moneylender had two debtors: one owed five hundred denarii, and the other fifty. When they were unable to repay, he graciously forgave them both. So which of them will love him more?" Turning toward the woman, He said to Simon, "Do you see this woman? I entered your house; you gave Me no water for My feet, but she has wet My feet with her tears and wiped them with her hair. You gave Me no kiss; but she, since the time I came in, has not ceased to kiss My feet. You did not anoint My head with oil, but she anointed My feet with perfume. For this reason I say to you, her sins, which are many, have been forgiven, for she loved much; but he who is forgiven little, loves little." (Luke 7:41, 42, 44-47 NASB)

Both Simon and the woman were sinners; he followed the law (at least outwardly, but perhaps with too much pride), while her sins were many. She had a repentant heart, and therefore sought forgiveness and was forgiven. Simon, who was following the law, lacked that faith we see in this woman, who risked a lot by entering the Pharisee's house to find Jesus.

So, from Bill Bray's sermon this morning at CCC (2) on Luke 7, are we going to have a religion response--characterized by a basis on pride in your good works, unawareness of your needs, and evidenced by activity apart from affections--or a repentant response--based on humility, aware of your needs, and evidenced by activity produced by affections?

Full text

Therefore you have no excuse, everyone of you who passes judgment, for in that which you judge another, you condemn yourself; for you who judge practice the same things. And we know that the judgment of God rightly falls upon those who practice such things. But do you suppose this, O man, when you pass judgment on those who practice such things and do the same yourself, that you will escape the judgment of God? Or do you think lightly of the riches of His kindness and tolerance and patience, not knowing that the kindness of God leads you to repentance? But because of your stubbornness and unrepentant heart you are storing up wrath for yourself in the day of wrath and revelation of the righteous judgment of God, who WILL RENDER TO EACH PERSON ACCORDING TO HIS DEEDS: to those who by perseverance in doing good seek for glory and honor and immortality, eternal life; but to those who are selfishly ambitious and do not obey the truth, but obey unrighteousness, wrath and indignation. There will be tribulation and distress for every soul of man who does evil, of the Jew first and also of the Greek, but glory and honor and peace to everyone who does good, to the Jew first and also to the Greek. For there is no partiality with God. (Romans 2:1-11 NASB)

Resources

  1. Luther, Martin. Luther's Commentary on Selected Bible Passages. Public Domain.
  2. www.cvillechurch.org
  3. Henry, Matthew. Matthew Henry's Commentary on the Whole Bible. Public Domain.
« Older posts Newer posts »

© 2018 Mininook

Theme by Anders NorenUp ↑