SW461-OL1 Syllabus,
“Pattern Recognition”


This programming intensive course introduces the student to the algorithms and data structures used in modern pattern recognition systems with an emphasis on those that can learn and improve their performance as they are used.  After a short review of some necessary mathematical concepts (Probability, Stochastic Processes and Vector Spaces), the student is introduced to the problem of representing real-world problems to a system.  Students will, in teams, design, build, test and report on major subsystems of a pattern recognition system.  Selected real world applications are used to show examples of some valid representations (e.g., Speech, Handwriting, EKG) to provide insight and experience in the application of recognition systems.  Several important recognition engines are then studied and analyzed for their effectiveness as the basis for recognition, synthesis, and learning systems.  The use of additional knowledge bases dealing with the problem environment is then introduced to increase system performance.

Instructor

Jeffrey N. Denenberg

Office location

https://Fairfield.zoom.us/j/7102932408

E-mail - Fairfield

jdenenberg@fairfield.edu

Zoom Office Hours

I'll be in my "office" whenever I am online, but just contact me and we can schedule a meeting,

E-Mail - Personal

jeffrey.denenberg@ieee.org

Google Voice

203-513-9427

Quickest Contact

Sending a private Quip chat message with a "@jdenenberg" is the quickest way to reach me as it alerts on my phone I will normally respond within a few hours, but not after 10 pm or before 9 am.

 

There will be a Zoom Q&A session most Sunday evenings so plan on joining our cohort at 8 PM on Sundays during the Summer session.

 


 

Course Objectives and Student Outcomes:

ABET CS Student Outcomes (1 - 5), and Bloom Cognitive Level in parenthesis

No.

Course Goals 

Student Learning Outcomes

1

Study the fundamental concepts and approaches to building pattern Recognition systems.

Students will be able to find solutions to complex problems that are poorly defined and under specified. 
(1, Synthesis)

2

Students will be able to apply several generic and application specific data reduction techniques on real world data.
(2, Application)

3

Study several classical and modern methods to classify input data as belonging to one of a set of predefined states and understand their relative strengths and weaknesses.

Students will be able use one or more of the studied recognition engines to classify data.
(2, Application, Analysis, Synthesis)

4

Work in small collaborative teams developing software in their preferred language to implement a number of data reduction algorithms that work on real world data.

Students will develop teaming skills that will allow them to work on large projects in the future.
(5, Analysis)

5

Students will demonstrate the use and application of their programming skills on advanced algorithms for analyzing "Big Data".
(2, Application, Analysis, Synthesis)

 


 

References

·       Pattern Classification”, Richard O. Duda, Peter E. Hart, and  David G. Stork, Wiley-Interscience; Ed 2 (November 9, 2000) , ISBN 978-0471056690. - This downloadable PDF of the classic text on this subject although it is dated, it is a great reference on many topics. The appendices include all the required mathematical backgrounds for studying Pattern Recognition.

·       Pattern Recognition and Machine Learning“, Christopher M. Bishop, Springer, ISBN 0-387-31073-8, (2006). This downloadable PDF is a later classic textbook on Pattern Recognition. I have the solutions to many of the end of chapter problems so I should be able help you if you try to do some of them.

·       “Mathematics for Machine Learning“, Marc Peter Deisenroth, Cambridge University Press, ISBN 110845514X, 2020 - Available in Paperback, Hard Cover, Kindle and free PDF versions. It covers the math background for most ML methods and requires very little previous math background to read.

·       “Pattern Recognition”, Sergios Theodoridis and Konstantinos Koutroumbas, Academic Press, Ed. 4, 2009, ISBN 978-1597492720. - A low cost text covering most of the topics in our course.

·       https://www.coursera.org/learn/machine-learning/

·       http://en.wikipedia.org/wiki/Machine_learning,www.sci.utah.edu/~gerig/CS6640-F2010/prob-tut.pdf

·       http://en.wikipedia.org/wiki/Variable-order_Markov_model

·       https://spectrum.ieee.org/consumer-electronics/audiovideo/deep-learning-reinvents-the-hearing-aid

Video Lectures

·       https://nptel.ac.in/courses/117108048/: Pattern Recognition - Prof. P. S. Sastry, Indian Institute of Science, Bangalore

·       https://www.youtube.com/playlist?list=PL17567A1A3F5DB5E4: Probability Primer MathematicalMonk on youtube

·       https://www.youtube.com/playlist?list=PL17567A1A3F5DB5E4Markov Models - MathematicalMonk on youtube

·       https://www.engineer4free.com/linear-algebra.htmlLinear-algebraEngineer4Free.com

Evidence of Learning

·       Each student collaborative team will develop and test their code on a sequence of mini-projects to demonstrate their acquired knowledge, skills and understanding.Through what performance tasks will students demonstrate the desired  knowledge, skills and understanding? (e.g., portfolios, written or visual texts, discussions, projects, etc.)

·       Each student will submit a “Reflection” on what was learned in the performance of each week’s project.

·       Team’s will produce a presentation describing selected projects and their results.

Assessment Strategy 

·       A Rubric will be filled out for each team project by each team member as well as the course leader
The other students will be asked to comment on each team’s approach to the problem.

·       A separate Rubric will be filled out for each “Reflection” by the course leader.

Course Grading

 

Item

Number

Weighting

Total

Mini Projects

4

10%

40%

Reflections on Mini Projects

4

5%

20%

Recognition Project

1

20%

20%

Reflection on Recognition Project

1

5%

5%

Class Participation

1

15%

15%

 

 

Total

100%

Instruction & Learning Experiences

·       What knowledge and skills will students need to achieve these outcomes?

o   They, as Computer Science or Data Sciences students should have the required programming skills.
(This course is programming language agnostic: Python, Matlab, Java, C/C++ ... )

o   They will have to acquire some Probability Theory and Linear systems knowledge from supplied resources to complete some of the assigned projects.

·       What activities will equip students with needed knowledge and  skills?

o   Their focused research to solve some of the problems that they encounter in completing the assigned projects.

o   Writing a reflection on their team’s activities and giving others feedback on their reflections

·       What materials and tools will best support students to acquire the  knowledge, skills and outcomes?

o   Their chosen Programming Integrated Development Environment (IDE)

o   Their ability to do effective searches for additional resources on the Internet.

Instructional Activities (Preliminary Breakdown)

·       Prior to class start 5/18/2020

o   Install Quip App(s) and familiarize yourself with Quip

o   Introduce yourself (record short video and insert it into the Intro document)

Part 1 of 4: Course Introduction

·       Week 1

o   Read and comment on course introduction, Quip, Course Procedures.

o   exchange personal contact information (emails, cellphone).

o   Establish small teams for projects.

o   Read and discuss “A Structured Approach to Pattern Recognition”.

Part 2 of 4: Reducing the Input Space (weeks 2-7)

·       Weeks 2-3 - Time domain segmentation 
Finding “critical points” and segment real data into smooth “strokes” (choose ECG or Handwriting data)

o   Read and comment on the suggested first mini-project topics as well as the supporting reference materials

o   Establish which first project topic assignment each group accepts.

o   Produce working code (include many comments in the source file to make it obvious what each portion of the code is supposed to do).

o   Upload the source code and example results of running the code to your team folder in a sub folder for this project.

o   Write a reflection on your team’s activity on the first mini-project.

o   Read and comment (suggestions for improvement, not critiques) on the other teams results.

o   Fill out the rubric sheet to evaluate your team’s performance on the first project.

·       Weeks 4-5 - Algorithms to reduce input space complexity

o   Complexity reduction of critical time segments

§  Vector Quantization, 

§  Low order Polynomial fit, 

§  Spline Graphics

o   An alternative approach: Wavelet Compression of data sequence 
(Two dimensional analysis of the input data: Time and Frequency)

·       Week 6 - Speech Data reduction (“cepstrum”, LPC, Wavelet Transform)

o   Definitions:

§  A cepstrum (/ˈkɛpstrʌm, ˈsɛp-, -strəm/; plural cepstra) is the result of taking the inverse Fourier transform (IFT) of the logarithm of the estimated signal spectrum in each time frame window.

§  Linear Predictive Coding (LPC)is a method for signal source modelling in speech signal processing. It is often used by linguists as a formant extraction tool.

§  Wavelet Transform - is similar to the Fourier transform (or much more to the windowed Fourier transform) with a completely different merit function. The main difference is this: Fourier transform decomposes the signal into sines and cosines, i.e. the functions localized in Fourier space; in contrary the wavelet transform uses functions that are localized in both the real (time) and Fourier space.

·       Week 7 - k-means clusteringaims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean, serving as a prototype of the cluster. ... Applying the 1-nearest neighbor classifier to the cluster centers obtained by k-means classifies new data into the existing clusters.

Part 3 of 4: Recognition Engines

·       Weeks :8-11 - Implement one of 4 “Recognition Engines”; Template Matching (AKA Dynamic Programming), Hidden Markov, Nth-Order Markov, Neural Networks

Part 4 of 4: Applying your work on real data

·       Weeks 12-13 - Use your recognition engine to classify the data you generated in the 2nd portion of this course. Use the results to improve your implementation and report on performance.

 

Types of Tech Tools Needed

·       By Instructor

o   Quip Apps - Already installed on my PC and Android phone

o   Zoom - already in use

o   MS Office - Already in use

o   A compatible Internet Browser (I have Chrome and Opera, both seem to work well)

·       By Students

o   They will need to install and get familiar with Quip before or in the first few days of the course

o   They will need to have one or more working IDEs for developing their project software (Matlab, Python, Java or C/C++ as the course is programming language agnostic).

o   A working Internet Browser (Firefox, Chrome, Safari, ...) (I have found that Opera also works)

o   The ability to create/edit/render several file types (PDF, MPEG4, DOCX)


 

Center for Academic Excellence - Policies and Resources

Religious Holidays


As indicated by Fairfield University’s policy, any student missing classes or other assigned work due to religious holiday observances shall have the opportunity to make up that work during the term. It is the student’s responsibility to notify the instructor in writing at least two weeks prior to any observed holiday.

Academic Integrity


It is expected that all work done for this class will be in strict compliance with the principles of academic honesty and integrity, as outlined in the Student Catalog. Cheating, plagiarism, copying, or dishonesty of any kind is not acceptable and will result in a zero for the exam or assignment and possible a failing grade for the class, and/or expulsion from the program.

Americans with Disabilities Act


Fairfield University is committed to achieving equal educational opportunities and providing students who have documented disabilities equal access to all University programs, services and activities. For this course to be accessible to all students, different accommodations or adjustments may need to be implemented. If you require accommodations for this course because of a disability, please contact the Office of Accessibility (OOA) as early as possible this semester as they are the designated department on campus responsible for approving and coordinating reasonable accommodations for students with disabilities. The Office of Accessibility will help you understand your rights and responsibilities under the American with Disabilities Act and Section 504 of the Rehabilitation Act and will provide you further assistance with requesting and arranging accommodations for your courses. Once you receive your accommodations letter, please make an appointment to meet with me as soon as possible so that we can review your accommodations together and discuss how best to help you achieve equal access in this course this semester.

Office of Accessibility, 203-254-4081, ooa@fairfield.edu