How to set JAVA_HOME when installing Oracle JDK on a Mac running OSX Mavericks:

How to set JAVA_HOME when installing Oracle JDK on a Mac running OSX Mavericks

Sourced from: https://www.blogger.com/home?pli=1

The right place to download the JDK for Java 7 is Java SE Downloads.

All the other links provided above, as far as I can tell, either provide the JRE or Java 6 downloads (incidentally, if you want to run Eclipse or other IDEs, like IntelliJ IDEA, you will need the JDK, not the JRE).

Regarding IntelliJ IDEA - that will still ask you to install Java 6 as it apparently needs an older class loader or something: just follow the instructions when the dialog pop-up appears and it will install the JDK 6 in the right place.

Afterwards, you will need to do the sudo ln -snf mentioned in the answer above:

sudo ln -nsf /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents \
    /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK

(copied here as it was mentioned that "above" may eventually not make sense as answers are re-sorted).

I also set my JAVA_HOME to point to where jdk_1.7.0_xx.jdk was installed:

export JAVA_HOME="/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home"

Then add that to your PATH:

export PATH=$JAVA_HOME/bin:$PATH

The alternative is to fuzz around with Apple's insane maze of hyperlinks, but honestly life is too short to bother.

Thanks to Marco at:

Marco
853513

Link to main response:

http://stackoverflow.com/a/19737307

Read a file, extract contents encapsulated in parentheses

Sourced from http://objectmix.com/awk/26995-retrieve-string-between-parentheses.html

$ cat some_file | gawk '{if (match($0,/\((.*)\)/,f)) print f[1]}' | cut -d ':' -f1 | sort | uniq

The above command will be helpful when reviewing stack trace output from command line to filter out and identify all the referenced .java files listed.

For example:

$ cat stacktrace_1.txt | gawk '{if (match($0,/\((.*)\)/,f)) print f[1]}' | cut -d ':' -f1 | sort | uniq
FilterFileSystem.java
Job.java
JobSubmissionFiles.java
JobSubmitter.java
Native Method
ProcessBuilder.java
RawLocalFileSystem.java
Shell.java
Subject.java
ToolRunner.java
UserGroupInformation.java

Extract a list of tarballs from the pwd into their own separate folders

•  Extract all .tar.gz files residing in current dir into their own folders based upon the tarball’s filename.
• Recommended naming convention for .tar.gz file is:  hostname-otherdetails.tar.gz (NOTE:  required details is the first hyphen which is used to separate the fields, using the first field as the target directory where the corresponding tarball will be extracted to
• To change the parsing separating character, change the hyphen reference in the cut -d portion of the statement to another character, such as an underscore— For example,  cut -d ‘_’ -f1 

$ for i in *.tar.gz; do mkdir `echo $i | cut -d '-' -f1`; tar xvfz $i -C `echo $i | cut -d '-' -f1`; done

Collecting MR1 task logs from TTs in a single command

$tar cvfzh /tmp/`hostname`-job_201407311402-tasklogs.tar.gz /var/log/hadoop-0.20-mapreduce/userlogs/job_201407311402_0001/*

Reason why we include the -h option in tar:

The following location typically includes symlinks to the actual location of the task logs, which typically resides in the working space of the TT:
/var/log/hadoop-0.20-mapreduce/userlogs/job_201407311402_0001/ 

$ls -l
lrwxrwxrwx 1 mapred mapred  81 Aug  8 07:54 attempt_201407311402_0001_m_000000_0 -> /mapred/local/userlogs/job_201407311402_0001/attempt_201407311402_0001_m_000000_0

Filter log file from common entries of no interest, along with blank lines

egrep -v "Trying to set|at "  hadoop-cmf-mapreduce1-JOBTRACKER-node.company.com.log.out | sed -e '/^\s*$/d' | less

Classic multi-platform Text Editor

I’ve been looking for a good classic CLI-version of a text editor that’s multi platform.

Apart from a heavily-customized version of Vim, I found this to be pretty awesome stuff:

http://triptico.com/software/mp.html

Some screenshots from the referring link above:


PS:  This is not my work, but the awesome work of Angel Ortega.  If you find this useful, please do support him and let him know!


Commands used to build on Ubuntu 14.04:

git clone https://github.com/angelortega/mp-5.x

git clone https://github.com/angelortega/mpdm

git clone https://github.com/angelortega/mpsl

sudo apt-get install libncursesw5-dev gnome-core-devel build-essential flex byacc lib ncursesw5 ncurses-dev ncurses-bin

cd mp-5.x
./config.sh --disable-gtk
make
sudo make-install

Multiply / Sum column data in awk/sed

Sourced from here:
http://unix.stackexchange.com/questions/115998/how-do-i-multiply-and-sum-column-data-using-awk-and-or-sed

I have several columns of data. I always have the same number of rows (say 5). In the 2nd column, I want to multipy the first value by 5, then the second value by 4, the third value by 3, etc. I then want to sum these values, and divide by the sum of the values in the second column. How would I do this in sed and/or awk?

Example:

4 5 7 1 2 3
5 1 2 3 1 2
4 2 3 6 1 2
3 4 1 6 3 3
2 3 1 2 1 6

Answer: (5*5 + 4*1 + 3*2 + 2*4 + 1*3)/(5 + 1 + 2 + 4 + 3) = 3.067

Solution 1:

Replace 6 with total (number of lines + 1) if needed:

awk '{mult+=$2*(6-NR); sum+=$2;} END {print mult/sum;}' yourfile.txt 

Displays: 3.06667

Solution 2:

If in case total number of lines are not known

$ cat file
4 5 7 1 2 3
5 1 2 3 1 2
4 2 3 6 1 2
3 4 1 6 3 3
2 3 1 2 1 6

$ awk 'FNR==NR{t = NR+1;next}{mult+=$2*(t-FNR);sum+=$2} 
       END{print mult/sum}' file{,}
3.06667

Explanation

awk 'FNR == NR{t=NR+1;next}

FNR variable --> Current line number in the current file

NR variable-->The total number of lines seen so far

The condition FNR == NR can only be true while awk is reading the first file of argument

t=NR+1 --> variable t holds line count +1 extra since it's required in our current context

next--> Stops the processing of the current input record and proceeds with the next input record

mult+=$2*(t-FNR) --> variable mult holds some of (5*(t-FNR) + ..... + 4*1 + 3*2 + 2*4 + 1*3)

sum+=$2 --> variable sum holds sum of column2

END--> AnEND rule is executed, once, after all the input has been read

print mult/sum --> Finally print mult/sum

file{,} --> you can also file file file but i used shortcut this bash trick you can say, try echo file{,} and echo file{,,} on your terminal.