Introduction
Introduction Statistics Contact Development Disclaimer Help
Reswizzle sigs -> classifiers, prep for real sigs - warvox - VoIP based wardial…
Log
Files
Refs
README
---
commit 4c5d46d2470eff1c43a8a7d088975c191330f06f
parent 917c87718c063533f4e01756585d8080ed2a4a5c
Author: HD Moore <[email protected]>
Date: Fri, 28 Dec 2012 20:04:10 -0600
Reswizzle sigs -> classifiers, prep for real sigs
Diffstat:
M app/views/home/about.html.erb | 14 +++-----------
A config/classifiers/.keep | 0
A config/classifiers/01.default.rb | 80 +++++++++++++++++++++++++++++…
A config/classifiers/99.default.rb | 36 +++++++++++++++++++++++++++++…
A config/signatures/.keep | 0
D config/sigs/01.default.rb | 97 ------------------------------
D config/sigs/99.default.rb | 56 -----------------------------…
M config/warvox.conf | 21 ++++++++++-----------
M lib/warvox/config.rb | 18 +++++++++---------
M lib/warvox/jobs/analysis.rb | 13 +++++--------
10 files changed, 143 insertions(+), 192 deletions(-)
---
diff --git a/app/views/home/about.html.erb b/app/views/home/about.html.erb
@@ -4,9 +4,9 @@
<h1 class='title'>About WarVOX</h1>
<b>WarVOX</b> is a product of <a href="http://www.rapid7.com/">Rapid7 Inc</a> …
-free software under the BSD license. WarVOX is intended for legal security ass…
+free software. WarVOX is intended for legal security assessment, asset invento…
and research purposes only. The latest version of WarVOX can be found at
-<a href="http://github.com/rapid7/warvox/">http://github.com/rapid7/warvox/</a…
+<a href="https://github.com/rapid7/warvox/">https://github.com/rapid7/warvox/<…
</td><td valign='top' align='center'>
@@ -17,7 +17,7 @@ and research purposes only. The latest version of WarVOX can …
<tr>
<td valign="top" align="right" class="header_item">
- WarVOX Version:
+ Version:
</td>
<td><%= WarVOX::VERSION %></td>
</tr>
@@ -76,14 +76,6 @@ and research purposes only. The latest version of WarVOX can…
<td><%= WarVOX::Conf %></td>
</tr>
-
-<tr>
- <td valign="top" align="right" class="header_item" width='200'>
- Data Storage:
- </td>
- <td><%= WarVOX::Config.data_path %></td>
-</tr>
-
<tr>
<td valign="top" align="right" class="header_item" width='200'>
Admin User:
diff --git a/config/classifiers/.keep b/config/classifiers/.keep
diff --git a/config/classifiers/01.default.rb b/config/classifiers/01.default.rb
@@ -0,0 +1,80 @@
+#
+# WarVOX Classifiers
+#
+
+#
+# These lightweight signatures are used to determine the
+# line type, which is the top-level classification that
+# differentiates between modem, fax, voice, and other
+# common results.
+#
+
+
+#
+# If you want to force your checks to run first, add your
+# logic to a file starting with "00." and place it in
+# this directory. Signature files are processed numerically
+# from lowest to highest (like RC scripts)
+#
+
+
+#
+# Initialize some local variables out of data
+#
+freq = data[:freq]
+fcnt = data[:fcnt]
+maxf = data[:maxf]
+
+#
+# Look for modems by detecting a 2100hz answer + 2250hz tone
+#
+if( (fcnt[2100] > 1.0 or fcnt[2230] > 1.0) and fcnt[2250] > 0.5)
+ @line_type = 'modem'
+ raise Completed
+end
+
+#
+# Look for modems by detecting a peak frequency of 2250hz
+#
+if(fcnt[2100] > 1.0 and (maxf > 2245.0 and maxf < 2255.0))
+ @line_type = 'modem'
+ raise Completed
+end
+
+#
+# Look for modems by detecting a peak frequency of 3000hz
+#
+if(fcnt[2100] > 1.0 and (maxf > 2995.0 and maxf < 3005.0))
+ @line_type = 'modem'
+ raise Completed
+end
+
+#
+# Look for faxes by checking for a handful of tones (min two)
+#
+fax_sum = 0
+[
+ fcnt[1625], fcnt[1660], fcnt[1825], fcnt[2100],
+ fcnt[600], fcnt[1855], fcnt[1100], fcnt[2250],
+ fcnt[2230], fcnt[2220], fcnt[1800], fcnt[2095],
+ fcnt[2105]
+].map{|x| fax_sum += [x,1.0].min }
+if(fax_sum >= 2.0)
+ @line_type = 'fax'
+ raise Completed
+end
+
+#
+# Dial tone detection (440hz + 350hz)
+#
+if(fcnt[440] > 1.0 and fcnt[350] > 1.0)
+ @line_type = 'dialtone'
+ raise Completed
+end
+
+#
+# To use additional heuristics, add new scripts to this directory
+# named XX.myscript.rb, where XX is a two digit number less than
+# 99 and greater than 01.
+#
+#
diff --git a/config/classifiers/99.default.rb b/config/classifiers/99.default.rb
@@ -0,0 +1,36 @@
+#
+# WarVOX Classifiers
+#
+#
+
+#
+# Initialize some local variables out of data
+#
+freq = data[:freq]
+fcnt = data[:fcnt]
+maxf = data[:maxf]
+
+# Look for voice mail by detecting the 1000hz BEEP
+# If the call length was too short to catch the beep,
+# this signature can fail. For non-US numbers, the beep
+# is often a different frequency entirely.
+if(fcnt[1000] >= 1.0)
+ @line_type = 'voicemail'
+ raise Completed
+end
+
+# Look for voicemail by detecting a peak frequency of
+# 1000hz. Not as accurate, but thats why this is in
+# the fallback script.
+if(maxf > 995 and maxf < 1005)
+ @line_type = 'voicemail'
+ raise Completed
+end
+
+
+#
+# Fall back to 'voice' if nothing else has been matched
+# This should be the last check processed
+#
+
+@line_type = 'voice'
diff --git a/config/signatures/.keep b/config/signatures/.keep
diff --git a/config/sigs/01.default.rb b/config/sigs/01.default.rb
@@ -1,97 +0,0 @@
-#
-# WarVOX Default Signatures
-#
-
-#
-# These signatures are used first and catch the majority of common
-# systems. If you want to force a different type of detection, add
-# your signatures to a file starting with "00." and place it in
-# this directory. Signature files are processed numerically from
-# lowest to highest (like RC scripts)
-#
-
-
-#
-# Initialize some local variables out of data
-#
-freq = data[:freq]
-fcnt = data[:fcnt]
-maxf = data[:maxf]
-
-#
-# Look for silence by checking for a strong frequency in each sample
-#
-scnt = 0
-ecnt = 0
-=begin
-freq.each do |fsec|
- scnt += 1
- if(fsec.length == 0)
- ecnt += 1
- next
- end
- sump = 0
- fsec.map {|x| sump += x[1] }
- savg = sump / fsec.length
- ecnt += 1 if (savg < 100)
-end
-=end
-
-# Store these into data for use later on
-data[:scnt] = scnt
-data[:ecnt] = ecnt
-
-#
-# Look for modems by detecting a 2100hz answer + 2250hz tone
-#
-if( (fcnt[2100] > 1.0 or fcnt[2230] > 1.0) and fcnt[2250] > 0.5)
- @line_type = 'modem'
- raise Completed
-end
-
-#
-# Look for modems by detecting a peak frequency of 2250hz
-#
-if(fcnt[2100] > 1.0 and (maxf > 2245.0 and maxf < 2255.0))
- @line_type = 'modem'
- raise Completed
-end
-
-#
-# Look for modems by detecting a peak frequency of 3000hz
-#
-if(fcnt[2100] > 1.0 and (maxf > 2995.0 and maxf < 3005.0))
- @line_type = 'modem'
- raise Completed
-end
-
-#
-# Look for faxes by checking for a handful of tones (min two)
-#
-fax_sum = 0
-[
- fcnt[1625], fcnt[1660], fcnt[1825], fcnt[2100],
- fcnt[600], fcnt[1855], fcnt[1100], fcnt[2250],
- fcnt[2230], fcnt[2220], fcnt[1800], fcnt[2095],
- fcnt[2105]
-].map{|x| fax_sum += [x,1.0].min }
-if(fax_sum >= 2.0)
- @line_type = 'fax'
- raise Completed
-end
-
-#
-# Dial tone detection (440hz + 350hz)
-#
-if(fcnt[440] > 1.0 and fcnt[350] > 1.0)
- @line_type = 'dialtone'
- raise Completed
-end
-
-#
-# To use additional signatures, add new scripts to this directory
-# named XX.myscript.rb, where XX is a two digit number less than
-# 99 and greater than 01.
-#
-#
-
diff --git a/config/sigs/99.default.rb b/config/sigs/99.default.rb
@@ -1,56 +0,0 @@
-#
-# WarVOX Fallback Signatures
-#
-#
-
-#
-# Initialize some local variables out of data
-#
-freq = data[:freq]
-fcnt = data[:fcnt]
-maxf = data[:maxf]
-ecnt = data[:ecnt]
-scnt = data[:scnt]
-
-
-# Look for voice mail by detecting the 1000hz BEEP
-# If the call length was too short to catch the beep,
-# this signature can fail. For non-US numbers, the beep
-# is often a different frequency entirely.
-if(fcnt[1000] >= 1.0)
- @line_type = 'voicemail'
- raise Completed
-end
-
-# Look for voicemail by detecting a peak frequency of
-# 1000hz. Not as accurate, but thats why this is in
-# the fallback script.
-if(maxf > 995 and maxf < 1005)
- @line_type = 'voicemail'
- raise Completed
-end
-
-=begin
-
-#
-# Look for silence by checking the frequency signature
-#
-if(freq.map{|f| f.length}.inject(:+) == 0)
- @line_type = 'silence'
- raise Completed
-end
-
-
-if(ecnt == scnt)
- @line_type = 'silence'
- raise Completed
-end
-=end
-
-#
-# Fall back to 'voice' if nothing else has been matched
-# This should be the last signature file processed
-#
-
-@line_type = 'voice'
-
diff --git a/config/warvox.conf b/config/warvox.conf
@@ -2,7 +2,6 @@
# WarVOX Configuration
#
-
#
# Configure the username and password for the WarVOX
# web interface. This password is sent in clear text
@@ -12,12 +11,6 @@ authentication:
pass: warvox
#
-# Configure the path to all saved data files
-# This requires ~500M of space per 10k prefix
-#
-data_path: "%BASE%/data/"
-
-#
# Configure filesystem paths to each required tool
#
tools:
@@ -26,9 +19,10 @@ tools:
iaxrecord: "%BASE%/bin/iaxrecord.rb"
#
-# Maximum processing jobs, normally this
-# is set to your processor core count,
-# but you can limit it further here.
+# Maximum processing jobs, normally this is set to your processor core count,
+# but you can limit it further here. Keep in mind that each analysis job also
+# requires at least 512Mb of RAM (based on audio capture length). A 300 second
+# audio capture could consume 3Gb of RAM per thread (the default is 53 seconds…
#
max_analysis_threads: 0
@@ -38,6 +32,11 @@ max_analysis_threads: 0
blacklist: "%BASE%/config/blacklist.txt"
#
+# Configure the classifier directory
+#
+classifiers: "%BASE%/config/classifiers"
+
+#
# Configure the signature directory
#
-signatures: "%BASE%/config/sigs"
+signatures: "%BASE%/config/signatures"
diff --git a/lib/warvox/config.rb b/lib/warvox/config.rb
@@ -32,13 +32,6 @@ module Config
)
end
- def self.data_path
- info = YAML.load_file(WarVOX::Conf)
- return nil if not info
- return nil if not info['data_path']
- File.expand_path(info['data_path'].gsub('%BASE%', WarVOX::Base…
- end
-
def self.analysis_threads
core_count = File.read("/proc/cpuinfo").scan(/^processor\s+:/)…
@@ -83,8 +76,15 @@ module Config
File.expand_path(info['signatures'].gsub('%BASE%', WarVOX::Bas…
end
- def self.signatures_load
- path = signatures_path
+ def self.classifiers_path
+ info = YAML.load_file(WarVOX::Conf)
+ return nil if not info
+ return nil if not info['classifiers']
+ File.expand_path(info['classifiers'].gsub('%BASE%', WarVOX::Ba…
+ end
+
+ def self.classifiers_load
+ path = classifiers_path
sigs = []
return sigs if not path
diff --git a/lib/warvox/jobs/analysis.rb b/lib/warvox/jobs/analysis.rb
@@ -13,7 +13,7 @@ class Analysis < Base
rescue ::LoadError
end
- class SignalProcessor
+ class Classifier
class Completed < RuntimeError
end
@@ -106,7 +106,7 @@ class Analysis < Base
fd.write(mr.audio)
end
- pfd = IO.popen("#{bin} '#{tmp.path}' '#{ dr.number.gsub("'", '…
+ pfd = IO.popen("#{bin} '#{tmp.path}' '#{ dr.number.gsub(/[^0-9…
out = Marshal.load(pfd.read) rescue nil
pfd.close
@@ -231,10 +231,10 @@ class Analysis < Base
end
#
- # Signature processing
+ # Classifier processing
#
- sproc = SignalProcessor.new
+ sproc = Classifier.new
sproc.data =
{
:raw => raw,
@@ -247,7 +247,7 @@ class Analysis < Base
:maxp => maxp
}
- WarVOX::Config.signatures_load.each do |sigfile|
+ WarVOX::Config.classifiers_load.each do |sigfile|
begin
str = File.read(sigfile, File.size(sigfile))
sproc.proc(str)
@@ -260,9 +260,6 @@ class Analysis < Base
# Save the guessed line type
res[:line_type] = sproc.line_type
- # Save any matched signatures
- res[:signatures] = sproc.signatures.map{|s| "#{s[0]}:#{s[1]}:#…
-
png_big = Tempfile.new("big")
png_big_dots = Tempfile.new("bigdots")
png_big_freq = Tempfile.new("bigfreq")
You are viewing proxied material from jay.scot. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.