Blog


Free Book: A Whirlwind Tour of Python

posted by MUHAMMAD MUN`IM AHMAD ZABIDI   [ updated ]


According to the author: "A Whirlwind Tour of Python is a fast-paced introduction to essential features of the Python language, aimed at researchers and developers who are already familiar with programming in another language. The material is particularly designed for those who wish to use Python for data science and/or scientific programming, and in this capacity serves as an introduction to my longer book, The Python Data Science Handbook."

About the author:
Jake VanderPlas Jake VanderPlas is a long-time user and developer of the Python scientific stack. He currently works as an interdisciplinary research director at the University of Washington, conducts his own astron‐ omy research, and spends time advising and consulting with local scientists from a wide range of fields. He is a co-author of the Python-powered text Statistics, Data Mining, and Machine Learning in Astronomy, and has presented many tutorials on the subject. He occasionally blogs about Python, Machine Learning, Visualization, and related topics at http://jakevdp.github.io.

Links:

Free Book: Think Python

posted Jun 20, 2019, 4:10 PM by MUHAMMAD MUN`IM AHMAD ZABIDI


Think Python is an introduction to Python programming for beginners. It starts with basic concepts of programming, and is carefully designed to define all terms when they are first used and to develop each new concept in a logical progression. Larger pieces, like recursion and object-oriented programming are divided into a sequence of smaller steps and introduced over the course of several chapters.

About the Author
  • Allen B. Downey (born May 11, 1967) is an American computer scientist, Professor of Computer Science at the Franklin W. Olin College of Engineering and writer of free textbooks. His GitHub repo is here.
Download Links


    Enthusiast Birding Lenses

    posted Jun 19, 2019, 1:00 AM by MUHAMMAD MUN`IM AHMAD ZABIDI   [ updated Jun 19, 2019, 11:05 PM ]

    Birding enthusiasts do it for fun, not for money. Therefore, they are always on the lookout for affordable equipment. Here is a list of affordable (<RM10k) birding lenses:

     Lens
    Focal Length 
     Compatibility
     Weight
    (g)
     Length
    (mm)
     Filter Size
    (mm)
     Price (RM) ex-Shashinki
    Canon EF 300mm f/4L IS USM
    300  C 1,190
    221
    77
     5938
    Nikkor 300mm f/4E PF ED VR AF-S
    300  N  755  148  77  6640
    M.Zuiko Digital ED 300mm f/4 IS PRO 300  O
     1,475  227
    77
     9494
    Canon EF 400mm f/5.6L USM
    400  C 1,250
    257
    77
     5268
    Canon EF 100-400mm f/4.5-5.6L IS II USM
    100-400  C  1,590  193
    77
     8539
    Nikkor 80-400mm f/4.5-5.6G AF-S ED VR
    80-400  N  1,563  203
     77  7745
    Nikkor 200-500mm F/5.6E ED VR AF-S 200-500  N  2,300  268
     95  4690
    Tamron SP 150-600mm f/5-6.3 Di VC USD G2
    150-600  CNS 2,010 260  95 4159
    Sigma 150-600mm f/5-6.3 DG OS HSM Contemporary 150-600  CN  1,950  260  95 4290
    Sony 200-600mm f/5.6-6.3 FE G OSS
    200-600  S  2,115  318  95  (USD2000)

    The Sony 200-600

    Notes:
    • The Sony 200-600 is not available locally yet.
    • Tamron & Sigma lenses are categorized as third-party.
    • Olympus, Tamron & Sigma make 100-400 lenses too. They cost less than RM4k but they don't seem too popular.


    R Result Analysis - Simple Reporting

    posted Jun 18, 2019, 12:14 AM by MUHAMMAD MUN`IM AHMAD ZABIDI   [ updated Jun 18, 2019, 8:51 PM ]

    Given a CSV file with the first 3 lines like this:

    $ head altera.csv
    name,matric,tests,final,mile1,mile4,prog,sect
    STUDENT 1,MATRIC1,9.1,31.5,14.25,15,2SKEE,1
    STUDENT 2,MATRIC2,12.35,22.5,13,9.25,2SKEE,1

    Read it and generate the total and grade columns. For CLO analysis, failure are not counted.


    > my=read.csv("altera.csv")
    > my$total <- round(my$tests + my$final +my$mile1 + my$mile4)
    > my$grade <- ifelse(my$total < 30, "E", ifelse(my$total<35, "D-", ifelse(my$total<40, "D", ifelse(my$total<45, "D+", ifelse(my$total<50, "C-", ifelse(my$total<55,"C", ifelse(my$total<60, "C+", ifelse(my$total<65, "B-", ifelse(my$total<70, "B", ifelse(my$total<75, "B+", ifelse(my$total<80, "A-", ifelse(my$total<90, "A", "A+"))))))))))))
    > table(my$grade)

     A A- A+  B B- B+  C C- C+ D+  E
    73 42 48 42 40 42 22  4 25  5  1


    The frame is filtered to remove failures.

    > x = my[my$total >=40,]
    > table(x$grade)

     A A- A+  B B- B+  C C- C+ D+
    73 42 48 42 40 42 22  4 25  5


    Columns clo1, clo2 and clo3 reports the students' achievement on the course learning outcomes normalized to 1.

    > x$clo1 <- (x$tests+x$final)/70
    > x$clo2 <- x$mile1/15
    > x$clo3 <- x$mile4/15
    > options(digits=2)
    > head(x,2)
          name  matric tests final mile1 mile4  prog sect total grade clo1 clo2 clo3
    1 STUDENT1 MATRIC1   9.1    32    14  15.0 2SKEE    1    70    B+ 0.58 0.95 1.00
    2 STUDENT2 MATRIC2  12.3    22    13   9.2 2SKEE    1    57    C+ 0.50 0.87 0.62

    CLO results for different sections: use the last 3 columns.

    > aggregate(x,by=list(x$sect),mean)
       Group.1 name matric tests final mile1 mile4 prog sect total grade clo1 clo2 clo3
    1        1   NA     NA    12    30    14    14   NA    1    70    NA 0.61 0.91 0.93
    2        2   NA     NA    12    31    14    14   NA    2    71    NA 0.62 0.94 0.90
    3        3   NA     NA    13    30    15    13   NA    3    71    NA 0.62 1.00 0.85
    4        4   NA     NA    12    29    15    14   NA    4    70    NA 0.58 0.99 0.94
    5        5   NA     NA    13    36    13     9   NA    5    71    NA 0.70 0.86 0.60
    6        6   NA     NA    15    40    14    13   NA    6    82    NA 0.79 0.90 0.85
    7        7   NA     NA    12    33    12    12   NA    7    68    NA 0.63 0.80 0.78
    8        8   NA     NA    11    31    12    12   NA    8    66    NA 0.60 0.82 0.80
    9        9   NA     NA    14    35    14    13   NA    9    77    NA 0.70 0.95 0.89
    10      10   NA     NA    14    35    14    14   NA   10    77    NA 0.70 0.94 0.96
    11      11   NA     NA    16    41    14    13   NA   11    83    NA 0.80 0.91 0.85
    12      12   NA     NA    12    32    14    13   NA   12    71    NA 0.63 0.91 0.87
    13      13   NA     NA    12    29    14    13   NA   13    68    NA 0.58 0.92 0.89

    There were 50 or more warnings (use warnings() to see the first 50)

    Ignore the warnings. We got all the numbers we need.
    CLO results for all sections:

    > mean(x$clo1)
    [1] 0.661
    > mean(x$clo2)
    [1] 0.921
    > mean(x$clo3)
    [1] 0.866


    Find how many students achieved KPI of 0.4:

    > sum(x$clo1>.4)
    [1] 314
    > sum(x$clo2>.4)
    [1] 343
    > sum(x$clo3>.4)
    [1] 329

    The academic programs differ by the 5th character in the prog column.

    > e <- subset(x,'E'==substr(x$prog,5,5))
    > mean(e$clo1)
    [1] 0.62
    > mean(e$clo2)
    [1] 0.94
    > mean(e$clo3)
    [1] 0.87

    The SKEL and SKEM has 3 and 4 subgroups, respectively. The SKEL and SKEM subsets can be extracted using the subset or filter function. Here two more ways to perform the same operation s are shown.

    > l <- filter(x, x$prog=="1SKEL" | x$prog == "2SKEL" | x$prog=="4SKEL")
    > m = subset(x, substr(x$prog,5,5) == 'M)

    The filter function does not seem to work all the time. Both filter and subset have subtle differences. Subset is part of the base R, filter is part the dplyr package. After subsetting, find the means.

    > mean(m$clo2)
    [1] 0.92

    etc...

    We can also perform subgroup analysis without first extracting a subset. We can just use the x dataframe directly. To find the mean for the variable 'clo1' for the 'SKEE' cohort,

    > options(digits=2)
    > mean(x$clo1)
    [1] 0.66
    > mean(x$clo1['E'==substr(x$prog,5,5)])
    [1] 0.62
    > mean(x$clo1['L'==substr(x$prog,5,5)])
    [1] 0.69
    > mean(x$clo1['M'==substr(x$prog,5,5)])
    [1] 0.68


    etc.. Repeat for CLO2 and CLO3.

    All the numbers required for reporting are now ready for presentation. Need to study ggplot  next.

    R Result Analysis - Preparing the Data

    posted Jun 17, 2019, 3:19 PM by MUHAMMAD MUN`IM AHMAD ZABIDI   [ updated Jun 18, 2019, 8:25 PM ]

    Given a CSV file with the first 3 lines like this:

    $ head altera.csv
    name,matric,tests,final,mile1,mile4,prog,sect
    STUDENT 1,MATRIC1,9.1,31.5,14.25,15,2SKEE,1
    STUDENT 2,MATRIC2,12.35,22.5,13,9.25,2SKEE,1

    Go into R, using Anaconda or Rstudio.
    Install dplyr (if not already installed) and import it.

    > install.packages(pkgs="dplyr")
    > library(dplyr)

    Read in the data file.

    > my=read.csv("altera.csv")

    Dump all data:

    > my

    Or just show the first 6 lines:

    > head(my)

           name    matric tests final mile1 mile4  prog sect
    1 student 1 MATRIC001  9.10  31.5 14.25 15.00 2SKEE    1
    2 student 2
    MATRIC002 12.35  22.5 13.00  9.25 2SKEE    1
    3 student 3
    MATRIC003 11.55  38.0 14.25 10.75 2SKEE    1
    4 student 4
    MATRIC004 11.60  28.0 14.25 10.75 2SKEE    1
    5 student 5
    MATRIC005  8.25  20.0 10.00 14.79 2SKEE    1
    6 student 6
    MATRIC006 15.30  38.5 15.00 15.00 2SKEE    1

    Make a table or two:

    > table(my$prog)

    1SKEE 1SKEL 1SKEM 2SKEE 2SKEL 2SKEM 3SKEM 4SKEL 4SKEM
        3     8     4   128   126    72     1     1     1

    > xtabs(~ prog + sect, my)
           sect
    prog     1  2  3  4  5  6  7  8  9 10 11 12 13
      1SKEE  0  0  1  0  1  0  0  0  0  0  0  1  0
      1SKEL  0  1  0  0  0  4  0  0  0  0  1  2  0
      1SKEM  1  0  0  0  0  0  0  0  0  0  0  3  0
      2SKEE 26 27 27 26 22  0  0  0  0  0  0  0  0
      2SKEL  0  0  1  1  0 27 20 14 28 30  0  5  0
      2SKEM  2  0  1  5  0  0  0  0  0  0 26  6 32
      3SKEM  0  1  0  0  0  0  0  0  0  0  0  0  0
      4SKEL  0  0  0  0  0  1  0  0  0  0  0  0  0
      4SKEM  0  0  1  0  0  0  0  0  0  0  0  0  0

    Create a total column

    > my$total <- my$tests + my$final +my$mile1 + my$mile4
    > my$total <- round(my$total)
    > head(my,2)

           name    matric tests final mile1 mile4  prog sect total
    1 student 1 MATRIC001  9.10  31.5 14.25 15.00 2SKEE    1    70
    2 student 2
    MATRIC002 12.35  22.5 13.00  9.25 2SKEE    1    57

    Create the passfail column:

    > my$passfail <- ifelse(my$total < 40, "fail", "pass")

    > head(my,2)

           name    matric tests final mile1 mile4  prog sect total passfail
    1 student 1 MATRIC001  9.10  31.5 14.25 15.00 2SKEE    1    70     pass
    2 student 2
    MATRIC002 12.35  22.5 13.00  9.25 2SKEE    1    57     pass

    Create the grade column:

    > my$grade <- ifelse(my$total < 30, "E", ifelse(my$total<35, "D-", ifelse(my$total<40, "D", ifelse(my$total<45, "D+", ifelse(my$total<50, "C-", ifelse(my$total<55,"C", ifelse(my$total<60, "C+", ifelse(my$total<65, "B-", ifelse(my$total<70, "B", ifelse(my$total<75, "B+", ifelse(my$total<80, "A-", ifelse(my$total<90, "A", "A+"))))))))))))

    Delete the passfail column, where 10 is the column number starting from 1:

    Delete the passfail column, where 10 is the column number starting from 1:

    > my <- my[,-10]

    > head(my,2)

           name    matric tests final mile1 mile4  prog sect total grade
    1 student 1 MATRIC001  9.10  31.5 14.25 15.00 2SKEE    1    70    B+
    2 student 2
    MATRIC002 12.35  22.5 13.00  9.25 2SKEE    1    57    C+

    Generate cross tables

    > xtabs(~ grade + sect, my)
         sect
    grade  1  2  3  4  5  6  7  8  9 10 11 12 13
       A   6  7  6  7  3 10  4  2  5  6  8  4  5
       A-  6  5  4  3  4  1  6  1  1  2  2  3  4
       A+  2  0  4  0  1 12  0  0  8  9 10  0  2
       B   3  2  1 10  4  2  1  3  4  3  1  3  5
       B-  2  4  1  3  3  1  3  5  1  3  3  5  6
       B+  2  7  6  4  4  2  1  1  5  5  2  1  2
       C   3  1  2  1  2  3  3  1  1  1  0  0  4
       C-  0  0  3  0  0  0  0  0  0  1  0  0  0
       C+  5  3  4  4  1  1  1  1  2  0  1  1  1
       D+  0  0  0  0  0  0  1  0  1  0  0  0  3
       E   0  0  0  0  1  0  0  0  0  0  0  0  0


    > xtabs(~ grade + prog, my)
         prog
    grade 1SKEE 1SKEL 1SKEM 2SKEE 2SKEL 2SKEM 3SKEM 4SKEL 4SKEM
       A      0     1     1    24    26    21     0     0     0
       A-     2     0     1    20    12     6     1     0     0
       A+     0     0     0     7    28    12     0     1     0
       B      0     2     1    18    13     8     0     0     0
       B-     0     4     0    11    16     9     0     0     0
       B+     0     1     0    23    13     5     0     0     0
       C      0     0     0     8     9     5     0     0     0
       C-     0     0     0     2     1     0     0     0     1
       C+     0     0     1    15     6     3     0     0     0
       D+     0     0     0     0     2     3     0     0     0
       E      1     0     0     0     0     0     0     0     0


    For the previous table, you can also use:

    > table(my$grade, my$sect)

    Then the summary table:

    > table(my$grade)

     A A- A+  B B- B+  C C- C+ D+  E
    73 42 48 42 40 42 22  4 25  5  1


    Until next time.


    Free Book: Fast Lane to Python

    posted Jun 16, 2019, 4:18 AM by MUHAMMAD MUN`IM AHMAD ZABIDI   [ updated Jun 16, 2019, 5:01 AM ]

    This book aims to enable the reader to quickly acquire a Python foundation. The material particularly feel quite comfortable to anyone with background in an object-oriented programming (OOP) language such as C++ or Java.
    Even if ones lack this background, they will still be able to read these sections, but will probably need to go through them more slowly than those who do know OOP. Some Linux knowledge would also be helpful, but it certainly is not required.
    Python is used on Windows and Macintosh platforms too, not just Linux. So, most statements here made for the Linux context will also apply to Macs as well. The author acknowledged that programming is a personal, creative activity, so everyone has his/her own view.
    It covers all essential Python knowledge. You can learn complete primary skills of Python fast and easily. The book includes practical examples for beginners and includes tests & answers for the college exam, the engineer certification exam, and the job interview exam.

    About the Author
    • Dr. Norm Matloff is a professor of computer science at the University of California at Davis, and was formerly a professor of statistics at that university. He is a former database software developer in Silicon Valley, and has been a statistical consultant for firms such as the Kaiser Permanente Health Plan. He was born and raised in the Los Angeles area, and has a PhD in pure mathematics from UCLA, specializing in probability/functional analysis and statistics.
    Download Links

    Exploring DNN Implementation Options

    posted May 27, 2019, 6:28 PM by MUHAMMAD MUN`IM AHMAD ZABIDI


    Taxonomy of DNN Acceleration Methods

    posted May 27, 2019, 6:27 PM by MUHAMMAD MUN`IM AHMAD ZABIDI


    My RAM

    posted Apr 29, 2019, 11:01 PM by MUHAMMAD MUN`IM AHMAD ZABIDI

    $ sudo dmidecode --type 17
    # dmidecode 3.1
    Getting SMBIOS data from sysfs.
    SMBIOS 2.7 present.

    Handle 0x002E, DMI type 17, 34 bytes
    Memory Device
        Array Handle: 0x002C
        Error Information Handle: Not Provided
        Total Width: 64 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: None
        Locator: DIMM0
        Bank Locator: BANK0
        Type: DDR3
        Type Detail: Synchronous Unbuffered (Unregistered)
        Speed: 1600 MT/s
        Manufacturer: Kingston    
        Serial Number: 6718B60
        Asset Tag: AssetTagNum0
        Part Number: 9905584-015.A00LF
        Rank: 2
        Configured Clock Speed: 800 MT/s

    Handle 0x0030, DMI type 17, 34 bytes
    Memory Device
        Array Handle: 0x002C
        Error Information Handle: Not Provided
        Total Width: 64 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: None
        Locator: DIMM1
        Bank Locator: BANK1
        Type: DDR3
        Type Detail: Synchronous Unbuffered (Unregistered)
        Speed: 1600 MT/s
        Manufacturer: Kingston    
        Serial Number: 781856E
        Asset Tag: AssetTagNum1
        Part Number: 99U5584-005.A00LF
        Rank: 2
        Configured Clock Speed: 800 MT/s


    Shortest Verilog Code

    posted Jan 21, 2019, 10:40 PM by MUHAMMAD MUN`IM AHMAD ZABIDI   [ updated Jan 21, 2019, 10:46 PM ]

    Guess what it does?

    module smart( input [2:0] x, output [7:0] y);
      assign y = 1 << x;
    endmodule

    It's a 3:8 decoder!

    The following is how it's done, normally:

    module naive(sel, res);
      input [2:0] sel;
      output [7:0] res;
      reg [7:0] res;

      always @(sel or res)
        begin
        case (sel)
          3'b000 : res = 8'b00000001;
          3'b001 : res = 8'b00000010;
          3'b010 : res = 8'b00000100;
          3'b011 : res = 8'b00001000;
          3'b100 : res = 8'b00010000;
          3'b101 : res = 8'b00100000;
          3'b110 : res = 8'b01000000;
          default : res = 8'b10000000;
        endcase
      end
    endmodule

    Now why would you want to do normal?

    1-10 of 107

    Comments