Data Analytics: Processing Images

Dev Fam,

This is pretty useful to know. Eventually, when I get the time, I'll look further into using this myself. But for now, just wanted to spread the news in case you haven't heard.

http://www.pcworld.com/article/3011757/developers-get-new-tools-from-google-to-analyze-images.html#tk.nl_today

http://thenextweb.com/google/2015/12/02/googles-new-cloud-vision-api-helps-teach-machines-to-better-understand-images/

Collecting data has always been a huge deal. Marketing terms like "big data" and "hadoop" have been constantly thrown around industries in attracting companies and customers. However, what's been difficult to analyze has been images collected. Thus, this makes such APIs as the ones mentioned in the article above a pretty powerful tool. Analyzing images and creating more metadata regarding the image and updating data storing appropriately to benefit the data analytical tools applied - will be very helpful in making better decisions shaping the future of people, companies, industries, and the world.

May the force be with you,
G2

Vagrant - Useful Commands

This is a list of useful commands to use with Vagrant. These commands are useful to know and saves time on searching. To see the argument options along with the command, click on the command.

Vagrant Reference URL

https://docs.vagrantup.com/v2/cli/index.html

Manage (add, remove, etc) virtual boxes

$vagrant box

// See the list of boxes installed into Vagrant.

$vagrant box list

// See any outdated boxes currently installed.

$vagrant box outdated

// Remove a box (and replace NAME_OF_BOX with the actual name of your box)

$vagrant box remove NAME_OF_BOX

// Update current box in Vagrant

$vagrant box update

Create, configure box

Create, configure guest machines according to local Vagrantfile.

$vagrant up

Suspend virtual box

Suspend guest machine in use (that Vagrant is managing) rather than shutting down or destroying it. This effectively saves the exact point-in-time state of the machine for resuming later (instead of full boot).
$vagrant suspend

Halt virtual box

Shuts down the running machine (box) Vagrant is managing. If graceful shutdown fails, then use argument 'force' to shut off completely.

$vagrant halt

$vagrant halt --force

$vagrant halt -f

Destroy virtual box

Stops the running machine (box) Vagrant is managing and destroys all resources created during the machine process. This leaves your machine in a clean state as if never used (like a reset button).
Use the 'force' argument to bypass the confirmation dialog.
$vagrant destroy
$vagrant destroy --force
$vagrant destroy -f

Reload virtual box

Halt followed by an Up.
$vagrant reload

Git - Useful Commands

This is a list of useful commands to use with Git. Once in a while, my Windows GitHub GUI app is not working as I would like and I need to do some back-end coding to speed things up. Other times, I just want to bypass using the GUI to speed up my coding or to automate processes. These commands are useful to know and saves time on searching.

Git Reference URL

https://git-scm.com/docs

Refresh my branch list (all branches including master)

- list both remote-tracking branches and local branches.

$git branch -a

Refresh my branch list

- list the remote-tracking branches.

$git branch --remotes
$git branch -r

Google: TensorFlow

Hey Dev Fam,

Are you into Machine Learning? Well, check this article out:
http://www.theverge.com/2015/11/9/9696020/google-machine-learning-tensorflow-open-source

Yep, and here's their blog about it:
https://googleblog.blogspot.com/2015/11/tensorflow-smarter-machine-learning-for.html

And, if you're the type that wants the white paper abstract:
http://download.tensorflow.org/paper/whitepaper2015.pdf

GitHub:
https://github.com/tensorflow/tensorflow

Website:
http://tensorflow.org/

To get started, go to the website (right above).

Cheers,
G2

Learning PHP - How to Begin and Win!

Dev Fam,

This is a good way to get started with learning PHP.

http://code.tutsplus.com/tutorials/the-best-way-to-learn-php--net-22287

The author, I think, has done a pretty good job with summing and articulating the steps to hit the track running as a PHP Developer.

Here are some book references as well.

• http://www.phpreferencebook.com/pdf/download/ (free e-Book)
• http://readwrite.com/2011/04/16/5-free-e-books-and-tutorials-f (list of more free e-Books)
• http://www.hackingwithphp.com/ (free online book)
• http://www.fromdev.com/2013/09/best-php-books.html (7 Best PHP Books)

Cheers,
G2

GitHub Auth Basics | Using Sinatra (RubyGem) | "Persistent" Authentication

Hi Dev Fam,

This is actually part 3 of executing the GitHub Basic Authentication tutorial. Find part 1 and part 2 that I did here:
http://g2coding.blogspot.com/2015/10/github-auth-basics-using-sinatra.html
http://g2coding.blogspot.com/2015/10/github-auth-basics-using-sinatra_20.html

Implementing "Persistent" Authentication

We add the files "advanced_server.rb" and "advanced.erb" (inside "views" folder). Let's make sure no other code is running this Sinatra server by simply opening browser and accessing port: 4567.

Now, let's run the code. Again, I'm using Windows OS. So, I'll open my command prompt, go to my Ruby project folder and run:
C:\Users\[username]\RubyProjects\github-auth> ruby advanced_server.rb

This is how the home page should display in browser.

Oops, we got an error. The problem relates to the advanced.erb file when accessing a "null" email variable. We'll make the update to the code as similar to the basic.erb (by adding code '!email.nil? &&')

Now, let's run again and see if we have implemented the persistent authentication (stored within our session object).

Victory! Yes, it works! Congratulations, you did it!

My Code

My code is exactly the same as in the GitHub tutorial for the "advanced_server.rb" file. However, I have a slight modification to the "advanced.erb" file. Here's how my code looks in the view.

What's Next?

Try using this authentication from an organization account instead of as a user.
https://developer.github.com/guides/managing-deploy-keys/

Duck-Typing

Dev Fam,

Here's something I just learned today - Duck Typing.

https://en.wikipedia.org/wiki/Duck_typing

The name of the concept refers to the duck test, attributed to James Whitcomb Riley, which may be phrased as follows:

When I see a bird that walks like a duck and swims like a duck and quacks like a duck, I call that bird a duck.^[1]

I can't believe I never heard of this. Of course, I think this may be from the rising of so many dynamic programming languages and its popular use nowadays.

I like this quote:
Criticisms around duck typing tend to be special cases of broader points of contention regarding dynamically typed versus statically typed programming language semantics.

Of course, if I'm programming with a statically typed language, then I'm not wasting time with this "duck-typing". I need to avoid assumptions and get straight to the point.

But, this is good to know. I encountered this while learning Ruby.

Peace,
G2

Java - abstract object getModifier(config) {...}

Hey my Dev Peeps,

So, I came across this line of Java code:

public AbstractEditor<? extends CoreConfig> getEditor(CoreConfig config) {...}

and was puzzled by the returned type being an extended object. I can't recall seeing this before. What does this mean? Why is this allowed in Java? Is this good coding?

Here's what I think.

The '?' is a wildcard <type> which allows any type to extend off the parent object (in this case, SourceConfig).

I like what this article response is specifically in referencing note from Effective Java:
http://stackoverflow.com/questions/18384897/what-is-difference-between-extends-object-and-e-extends-object

I also read this article which I think has some great, simple insight:
http://stackoverflow.com/questions/9646397/does-php-have-an-answer-to-java-style-class-generics

I'll need to investigate this more to conclude my quest...

Semantica Pro - remote-config (.xml) and Preferences (.conf)

Hey Fam,

I'm still learning more about this nice tool "Semantica". Today, I'm learning a lot more on its configurations which is very helpful if you're developing plug-ins for Semantica.

I'll highlight two files today: 1) remote-config.xml, 2) Preferences.conf.

remote-config.xml

CAUTION: Semantica might delete this file and its subfolders when it starts up because the application settings are looking for another set of configuration files. All of these files in the "C:/Users/[username]/Semantica/RemoteConfiguration" will be deleted. Hence, it might be wise to keep a copy somewhere within Semantica or where these files can be recovered when accidents occur.

When setting up Semantica initially, there is a way to point to a default configuration file that helps configure Semantica's display and functionality access for the user. These typically go into the "remote-config.xml" file. This file, along with its subfolders, are usually stored in the path "C:/Users/[username]/Semantica/RemoteConfiguration". Here's a quick screenshot, but I'm highlighting here its importance to remove the default Sources (data sources that Semantica connects to retrieve data). Later, we can add data sources that are in respects to our project(s).

When Semantica loads (i.e. starts up to display to user), this application looks for these configurations. An option is presented to the user if this configuration file is not found. The option asks whether to continue using Semantica without one (configuration file) or with one (which will need to be added/updated).

To avoid this dialog and know the back-end settings which trigger this dialog upon loading of application, we must update the application's vmptions which will update the "Preferences.conf" file.

Semantica Professional.vmoptions

This is one of the core files in loading the application (similar to Eclipse IDE's eclipse.ini file).
Within this application is where I set the path to my remote-config file. This will overwrite the path stated in the Preferences.conf file automatically upon application start-up.

Preferences.conf

Semantica will overwrite this file every time the application is loaded with its latest settings (of preferences).

Updating path to remote-config

I'm going to update my value for RemotePreferences.URL to
"file\:///C\:/Users/gordonr/Semantica/RemoteConfiguration/remote-config.xml" and add in my remote-config.xml along with its subfolders. Next, I'll restart my Semantica and expect no dialog to display.

Unfortunately, I still got the error and then when I clicked on "Update" I received this error message:

I guess, the only way to update this path to the remote-config.xml is when we are actually in the application. So, let's just enter the application, select on the menu's File > Preferences > General > scroll down general settings pane to "Remote Configuration Settings" > update path to "remote-config.xml". Copy the "remote-config.xml" and its subfolders back into the RemoteConfiguration folder. Restart Semantica.

Sigh. We still get the RemoteConfiguration folder emptied by Semantica and I have no reason or understanding of why. So, we'll just copy the "remote-config.xml" and its subfolder into a new folder. This will be in "C:/Users/[username]/Semantica/configuration". Yep, keep it simple. Next, go into the file "Semantica Professional.vmptions" under Semantica Professional app directory and update the path to our new configuration folder. Now, let's restart Semantica and see what happens.

Interesting, the vmoptions folder didn't update this Preferences.conf file which also means this application didn't update it's URL for the remote configuration settings. And I'm really feeling like I'm wasting a lot of time here. Some things seem complex for no reason. I'll try updating the preferences from within the File > Preferences again.

This time, see the Remote Configuration file is stated as invalid. Update the path to where the remote-config.xml is located. Then, click on "Update" button.

Notice a dialog displays asking for user to confirm overwriting of config file. Overwrite. Next, see a dialog stating the config will take effect after restart of application. Let's go ahead and restart the application. No need to save changes (i.e. select "No" when asked to save changes because we didn't make any network, entity, or oncology changes).

When Semantica restarts, we see confirmation messages regarding the updates to remote configurations.

Select OK when asked regarding these updates. Now when we go into the Import Manager view, we'll see some very general data sources that actually don't do anything.

Also, notice in the "Preferences.conf" file that the updates are also applied. Strangely, I experimented with the vmoptions file and didn't see any auto-updates in there related to the restart or config of Semantica. I'm doubting my understanding of this file's usage (i.e. vmoptions). And I can't believe I have to set my config path within Semantica's general settings.

I feel unsuccessful today, but I learned a lot. I hope you did, too.

Semantica Pro - quick, very basic overview

Hello Dev Fam,

Today I'm learning about this Semantica Professional application. It's an application that retrieves data from various sources (i.e. data sources), allows a user to aggregate the data, allows a user to view a summary of records retrieved, allows a user to create some reports from the data retrieved, and has many other functions. It's primary use is to create ontology reports. I think.

My team is creating plug-ins (libraries to be added into this application which shape how the application looks and can be used) for Semantica. Some of these plug-ins are very large and complex. Others are simple and small. It's a pretty nice tool, but there is a steep learning curve to using this tool. In addition, there is a steep learning curve in coding this tool. This seems to be a Java Enterprise application and plenty of JavaBeans are used.

Here's the beautiful the entry screen.

Here's the application's user environment. This is the application and it's like a reporting tool but with many powerful features to search for information on various data sources and configure report views.

I can't go into further details of this application since I didn't develop this and would like to be respectful to the vendor (and developers). But, this is a very nice tool! I just wish they had some quick web tutorials or video clips within the installation to help people like me learn this application much quicker.

References:

http://www.semanticresearch.com/
http://www.semanticresearch.com/semantica-pro
http://semanticweb.org/wiki/Main_Page.html

GitHub Auth Basics | Using Sinatra (RubyGem) | Installing Ruby - Part 2

Fam,

Ok, so I struggled yesterday with trying to get my simple Ruby application running in Sinatra locally according to GitHub's Authentication tutorial. See my struggle here on Part 1.

So, I had to start over and ensure my understanding of what was happening on my machine. This happened earlier today with reinstalling Ruby and Sinatra. See my victory and coding pleasure here. Lots of thanks to those who created the RubyInstaller project and gave great instructions on their GitHub wiki.

Now, I'm going to see how to continue my path with the GitHub Authentication tutorial.

I want to see if my Ruby application works now if I run "server.rb". This file is in my directory located at "C:\Users\[me]\RubyProjects\github-auth". I have a subfolder in here as well named "views" which contains the file "index.erb".

Code for "server.rb":
require 'sinatra'
require 'rest-client'
require 'json'

CLIENT_ID = ENV['GH_BASIC_CLIENT_ID']
CLIENT_SECRET = ENV['GH_BASIC_SECRET_ID']

get '/' do
  erb :index, :locals => {:client_id => CLIENT_ID}
end

Code for "index.erb":
<html>
  <head>
  </head>
  <body>
    <p>
      Well, hello there!
    </p>
    <p>
      We're going to now talk to the GitHub API. Ready?
      <a href="https://github.com/login/oauth/authorize?scope=user:email&client_id=<%= client_id %>">Click here</a> to begin!</a>
    </p>
    <p>
      If that link doesn't work, remember to provide your own <a href="/v3/oauth/#web-application-flow">Client ID</a>!
    </p>
  </body>
</html>

Installing the Gems

First, I know I need to ensure I have the libraries installed via "gem" considering they're called out in my "server.rb" file. Here are the libraries (or gems) that I'm talking about:

• sinatra
• rest-client
• json

Here are the commands I enter in the Windows Command Prompt (but I'm skipping sinatra and json because I already installed these from previous blogs above):
> gem install sinatra --platform=ruby
> gem install json --platform=ruby
> gem install rest-client --platform=ruby

Here's the result:

Wow! No issue at all. Again, notice we're installing all the gems on Windows via Command Prompt (avoiding Cygwin).

Ok, let's go to the directory with the "server.rb" file and run it via Command Prompt. Oh no, I'm getting an error.

Troubleshooting a Gem (rest-client)

From searching for solutions on the Internet, I realized from this SO question (http://stackoverflow.com/questions/7964778/no-such-file-to-load-ffi-c-loaderror) that I might not have the library (gem) installed called "ffi". So, I run the following command:

>gem install ffi --platform=ruby

Yes, this resolved my issue!

Now we are becoming troubleshooting experts with Ruby while learning this Basic Authentication in GitHub - woohoo! Finally making progress.

GitHub Basic Authentication - Coding Foward

Now that our Sinatra application works, let's click on the "Click here" and see if we can continue to follow GitHub's instructions.

Since I didn't sign in yet, GitHub has a nice security setting that ensures I'm logged in before proceeding. I add in my credentials and sign in.

Yes, finally we are making progress with the GitHub Basic Authentication tutorial. The nice thing here is that we already added our GitHub Client ID and Client Secret into our GitHub stored application and as environment variables on my local machine. So when we did click on the link "Click here", the Sinatra server executed the code and automatically connected everything (values and online connection).

Now we need to continue with instructions from section "Providing a Callback".

Providing a Callback

I update my file "server.rb" according to the GitHub instructions and now my code looks like this:

require 'sinatra'
require 'json'
require 'rest-client'

CLIENT_ID = ENV['GH_BASIC_CLIENT_ID']
CLIENT_SECRET = ENV['GH_BASIC_SECRET_ID']

get '/' do
  erb :index, :locals => {:client_id => CLIENT_ID}
end

get '/callback' do
  # get temporary GitHub code...
  session_code = request.env['rack.request.query_hash']['code']
  
  # ... and POST it back to GitHub
  result = RestClient.post('https://github.com/login/oauth/access_token',
                      {:client_id => CLIENT_ID,
                       :client_secret => CLIENT_SECRET,
                       :code => session_code
                      },
                      :accept => :json)
  
  # extract the token and granted scopes
  access_token = JSON.parse(result)['access_token']
end

I restart my Sinatra (local ruby server) and re-run the "server.rb" file in order for the new code to take effect. In case, we forgot - Ctrl+C, then >ruby server.rb.

Excellent! We received a token from GitHub's API Authentication.
332bc61cc3579befef5cac7d9003350c51c7a870

Checking Granted Scopes

I now attempt the next section and add the code for checking the user scope. Strange. I get another error (to troubleshoot).

I had to update the code so I could understand it more (see final code below in screenshots). This took me some time to troubleshoot because I had to learn Ruby code to troubleshoot. =p Once fixed, this is the resulting screen:

Yep, a blank screen.

Making Authenticated Requests

Now that we have the scope granted for my user, let's access my private email. Notice, I check my email settings to be certain of my settings. I check in my personal settings and in my application's settings. I don't care about the generated tokens because my code will generate one based on my client ID and Secret related to my application called "basics-of-authentication".

So, I should be able to access my email address once my access is authenticated. I add in the code per GitHub's tutorial. Ugh, another error.

GitHub's authentication is so irritating. I'm running into another issue but this time related to my user authentication (user permissions because I'm able to get my user scope). Here's another way I tried checking (by pasting the line of code directly into the browser but without my token).

They're similar messages. So, my token must not be the issue. After quite a lot of troubleshooting, I finally updated the code (to by-pass getting the authenticated user information and instead getting user's public information). Notice this JSON results parses into a Hash Map.

Then, I get the emails and add into the Hash Map at key 'private emails'. Finally, pass the authenticated results as variable locals to the basic.erb file. This smooth flow finally displays the following screen.

Victory at last!

Here's an update that might be useful to know and I'm keeping for my records. :)
https://developer.github.com/changes/2014-04-08-reset-api-tokens/

My Code:  server.rb

Enjoy!

GitHub Auth Basics | Using Sinatra (RubyGem) | Installing Ruby

I want to follow this tutorial to understand how to further implement or use this GitHub Authentication. https://developer.github.com/guides/basics-of-authentication/

However, seems I need to use a quick Sinatra application. https://github.com/sinatra/sinatra-book/blob/master/book/Introduction.markdown#hello-world-application

I'm not sure what this is. In following the README file (or markdown), I'll need to install RubyGems.
https://rubygems.org/

To use RubyGems, seems I'll need to install Ruby.
https://www.ruby-lang.org/en/

To install Ruby on my local Windows machine, seems it'll be easiest with the Installer.
http://rubyinstaller.org/downloads/

While installing Ruby, I selected all 3 checkboxes (using ToolKit for GUI development, adding Ruby exe to PATH, and auto-launching .rb and .rbw files with Ruby). Finish the installation.

Here's where I installed my Gem and Ruby programs.

Here's how I checked my programs were installed correctly. Checking Ruby first, then RubyGems.

I most likely will want to leverage the Unix guidelines from GitHub. Since I'm on Windows, I'll use the Cygwin program tool.
NOTE: If you already have this program open, then you'll probably need to close and reopen the program so that Cygwin will refresh with the system environment variables.

Interesting, I ran into an issue where 'gem' command is not found in Cygwin. I found these articles to help me understand why.
http://stackoverflow.com/questions/3831131/rubygems-cygwin-posix-path-not-found-by-ruby-exe
http://blog.mmediasys.com/2008/10/27/handy-tip-dont-mix-one-click-installer-with-cygwin/

So, I install Ruby from within the Cygwin setup.

Run the commands again to see if RubyGems is now properly installed within Cygwin.
$ gem

Yes, fixed!

Now, I go back to GitHub's tutorial to continue (from Installation section).

Since I was a little confused on what all to install to get a basic Sinatra app running, I followed the instructions based on this quick tutorial. Mind you, I didn't know Ruby and had to watch the quick "Ruby for Newbies" screencast.
http://code.tutsplus.com/tutorials/singing-with-sinatra--net-18965

I created my "basics.rb" file in my new folder called "RubyProjects".

Next, I ran my file with Ruby from within my Cygwin (which I changed to RubyProjects directory).

Now I see the results in my browser as mentioned in the tutorial.

I'm not going to finish this 'tutsplus' tutorial since I understand the basics now. I' now I go back to GitHub's tutorial to continue (from Installation section). I update my file to look similar to the one in GitHub's tutorial.

To see these changes take effect, I enter command (while focus in Cygwin window):  Ctrl + C
And the Sinatra server should stop (i.e. shutdown).

Next, I re-run the same command ($ ruby basics.rb) and see the update in the browser (refreshed).

Yes, we finished this simple instruction/tutorial from the referenced Sinatra on GitHub.
https://github.com/sinatra/sinatra-book/blob/master/book/Introduction.markdown#hello-world-application

Now what? Let's look back at:  https://developer.github.com/guides/basics-of-authentication/

Seems GitHub tutorial is stating this is my app. Most likely it's referring to this app being registered in GitHub. So, I'll need to commit this app in my GitHub and then register this app.

For this example, I'll just create a public repo in my GitHub. (Be sure to add .gitignore for Ruby.)

Here it is @ https://github.com/gradney/basics-of-authentication
I'll clone this repo to my local, copy the "basics.rb" file into my local clone, update the file name to "server.rb" as mentioned in the tutorial, and then push/commit this up to my online GitHub repo.

I update the "server.rb" with the code in the section "Accepting User Authorization".
https://developer.github.com/guides/basics-of-authentication/#accepting-user-authorization

I also add the subfolder "views" with the file "index.erb" into my cloned folder. I commit to online repo.

I register my application according to GitHub tutorial.

In Cygwin, I navigate to the project folder and attempt to start this application in Sinatra server. But, didn't work. I stop and restarted, and still didn't work. One of my imports ('require' libraries is not recognized). I'll need to install this gem.

$ gem install rest-client

I get an error stating I need to install the development tools first. :(

I tried the following command, but this didn't work.

$ gem install bundler 

I found this online article and tied this command

$ curl -sSL https://get.rvm.io | bash -s stable

Still does not seem to be working.

I just might need to build my own native libraries. boo.

https://github.com/oneclick/rubyinstaller/wiki/Development-Kit#building-the-devkit

$ gem install json_pure   (successful!)

$ gem update (there are so many errors resulting from "The compiler failed to generate an executable file.")

Yep, I'm going to need to do something else here with Sinatra. Dang, why does GitHub only have a tutorial on GitHub Authentication using a quick app in Ruby (via Sinatra)??? Awwww.....being difficult. Use an app that's platform independent in tutorials....that's the lesson today.

Some helpful troubleshooting tips:

https://github.com/oneclick/rubyinstaller/wiki/Troubleshooting

http://collaborate.je/2014/08/setting-ruby-rails-windows-7-via-cygwin/
http://stackoverflow.com/questions/20688671/failed-to-build-gem-native-extension-on-windows-7-the-system-cannot-find-the
https://www3.ntu.edu.sg/home/ehchua/programming/cpp/gcc_make.html