Installing CRITS (Collaborative Research Into Threats) on Ubuntu 16.04 LTS x64 Part#1

CRITs is an open source malware and threat repository that leverages other open source software to create a unified tool for analysts and security experts engaged in threat defense. It has been in development since 2010 with one goal in mind: give the security community a flexible and open platform for analyzing and collaborating on threat data. CRITs can be installed locally for a private isolated instance or shared among other trusted organizations as a collaborative defense mechanism.

Issue
CRITS can easily be installed via bootstrap, but for a more permanent deployment it is recommended that you follow this guide. Unfortunately, if you do, you’ll need to do a lot of extra research in order to make it “usable”; those steps are covered in the below solution:

Solution
Download Ubuntu Server 16.04.3 LTS
Create your Server VM and install Ubuntu Server 16.04.3 LTS (with OpenSSH Server if you require remote access)
Once your system is up and running, ssh/console and run update / install pre-requisites:

sudo apt update && sudo apt upgrade -y

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/mongodb.list
sudo apt update
sudo apt autoremove -y
sudo mkdir -pv /data/{db,logs}
sudo apt install -y --fix-missing automake libtool libchm-dev libchm1 libimage-exiftool-perl libarchive-zip-perl antiword libcairo2 libjbig0 liblcms2-2 libpixman-1-0 libpoppler58 libtiff5 poppler-utils libjbig-dev liblzma-dev libopenjpeg5 libpng12-dev libtiffxx5 libwebp5 libwebpdemux1 libwebpmux1 libjansson4 yara clamav-base clamav-freshclam libclamav7 libcurl3 libllvm3.6v5 apache2 build-essential tshark libnet1-dev libevent-dev zlib1g-dev libjpeg-dev libfuzzy-dev libldap2-dev libpcap-dev libpcre3-dev libsasl2-dev libssl-dev libtool libxml2-dev libxslt1-dev libyaml-dev mongodb-org numactl p7zip-full python-dev python-pip ssdeep swig upx-ucl zip libapache2-mod-wsgi fontconfig fontconfig-config fonts-dejavu-core geoip-database-extra gstreamer1.0-plugins-base javascript-common libasound2 libasound2-data libasyncns0 libauthen-sasl-perl libavahi-client3 libavahi-common-data libavahi-common3 libboost-filesystem1.58.0 libboost-system1.58.0 libc-ares2 libcapnp-0.5.3 libcdparanoia0 libcups2 libdrm-amdgpu1 libdrm-intel1 libdrm-nouveau2 libdrm-radeon1 libegl1-mesa libencode-locale-perl libevdev2 libfile-basedir-perl libfile-desktopentry-perl libfile-listing-perl libfile-mimeinfo-perl libflac8 libfont-afm-perl libfontconfig1 libfontenc1 libgbm1 libgl1-mesa-dri libgl1-mesa-glx libglapi-mesa libgraphite2-3 libgstreamer-plugins-base1.0-0 libgstreamer1.0-0 libgudev-1.0-0 libharfbuzz0b libhtml-form-perl libhtml-format-perl libhtml-parser-perl libhtml-tagset-perl libhtml-tree-perl libhttp-cookies-perl libhttp-daemon-perl libhttp-date-perl libhttp-message-perl libhttp-negotiate-perl libice6 libinput-bin libinput10 libio-html-perl libio-socket-ssl-perl libipc-system-simple-perl libjs-openlayers libllvm4.0 liblua5.2-0 liblwp-mediatypes-perl liblwp-protocol-https-perl libmailtools-perl libmirclient9 libmircommon7 libmircore1 libmirprotobuf3 libmtdev1 libnet-dbus-perl libnet-http-perl libnet-smtp-ssl-perl libnet-ssleay-perl libnghttp2-14 libnl-route-3-200 libogg0 libopus0 liborc-0.4-0 libpciaccess0 libprotobuf-lite9v5 libproxy1v5 libpulse0 libqgsttools-p1 libqt5core5a libqt5dbus5 libqt5gui5 libqt5multimedia5 libqt5multimedia5-plugins libqt5multimediawidgets5 libqt5network5 libqt5opengl5 libqt5printsupport5 libqt5svg5 libqt5widgets5 libsbc1 libsensors4 libsm6 libsmi2ldbl libsndfile1 libssh-gcrypt-4 libtheora0 libtie-ixhash-perl libtimedate-perl libtxc-dxtn-s2tc0 liburi-perl libvisual-0.4-0 libvorbis0a libvorbisenc2 libwacom-bin libwacom-common libwacom2 libwayland-client0 libwayland-server0 libwireshark-data libwireshark8 libwiretap6 libwscodecs1 libwsutil7 libwww-perl libwww-robotrules-perl libx11-protocol-perl libx11-xcb1 libxaw7 libxcb-dri2-0 libxcb-dri3-0 libxcb-glx0 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-present0 libxcb-randr0 libxcb-render-util0 libxcb-render0 libxcb-shape0 libxcb-shm0 libxcb-sync1 libxcb-util1 libxcb-xfixes0 libxcb-xkb1 libxcomposite1 libxcursor1 libxdamage1 libxfixes3 libxft2 libxi6 libxinerama1 libxkbcommon-x11-0 libxkbcommon0 libxml-parser-perl libxml-twig-perl libxml-xpathengine-perl libxmu6 libxpm4 libxrandr2 libxrender1 libxshmfence1 libxt6 libxtst6 libxv1 libxxf86dga1 libxxf86vm1 qttranslations5-l10n wireshark wireshark-common wireshark-qt x11-common x11-utils x11-xserver-utils xdg-utils
sudo ldconfig

Update MongoDB ownership, log/storage location, and create service :

sudo sed -i 's/dbpath=\/var\/lib\/mongodb/dbpath=\/data\/db/' /etc/mongod.conf
sudo sed -i 's/logpath=\/var\/log\/mongodb\/mongod.log/logpath=\/data\/logs\/mongod.log/' /etc/mongod.conf
sudo chown mongodb /data/db /data/logs
sudo bash -c 'cat <<EOF> /etc/systemd/system/mongod.service
[Unit]
Description=High-performance, schema-free document-oriented database
After=network.target
[Service]
User=mongodb
ExecStart=/usr/bin/mongod --quiet --config /etc/mongod.conf
[Install]
WantedBy=multi-user.target
EOF'

sudo systemctl enable mongod
sudo systemctl start mongod

Configure and install CRITS:

sudo useradd crits
cd /data
sudo git clone https://github.com/crits/crits.git
cd /data/crits/
sudo -H pip install -r requirements.txt
sudo touch /data/crits/logs/crits.log
sudo ln -f -s /data/crits/logs/crits.log /data/logs/crits.log
sudo chmod 664 /data/crits/logs/crits.log
sudo usermod -a -G crits www-data
sudo chgrp -R crits logs
sudo cp /data/crits/crits/config/database_example.py /data/crits/crits/config/database.py
SC=$(cat /dev/urandom | LC_CTYPE=C tr -dc 'abcdefghijklmnopqrstuvwxyz0123456789!@#%^&*(-_=+)' | fold -w 50 | head -n 1)
SE=$(echo ${SC} | sed -e 's/\\/\\\\/g' | sed -e 's/\//\\\//g' | sed -e 's/&/\\\&/g')
sudo -H sed -i -e "s/^\(SECRET_KEY = \).*$/\1\'${SE}\'/1" /data/crits/crits/config/database.py

Configure Apache & SSL (create self signed cert):

sudo /etc/init.d/apache2 stop
sudo rm -rf /etc/apache2/sites-available/*
sudo cp /data/crits/extras/*.conf /etc/apache2
sudo cp -r /data/crits/extras/sites-available /etc/apache2
sudo rm /etc/apache2/sites-enabled/*
sudo ln -f -s /etc/apache2/sites-available/default-ssl /etc/apache2/sites-enabled/default-ssl
sudo sed -i 's/Include \/etc\/apache2\/conf.d\//#Include \/etc\/apache2\/conf.d\//' /etc/apache2/apache2.conf
sudo sed -i "/export\ LANG\=C/ s/C/en\_US\.UTF\-8/" /etc/apache2/envvars
cd /tmp
openssl req -nodes -newkey rsa:4096 -keyout new.cert.key -out new.cert.csr -subj "/CN=CRITS/O=LOCAL/C=US"
openssl x509 -in new.cert.csr -out new.cert.cert -req -signkey new.cert.key -days 1825
sudo cp new.cert.cert /etc/ssl/certs/crits.crt
sudo cp new.cert.key  /etc/ssl/private/crits.plain.key
sudo a2enmod ssl
service apache2 restart

Create CRITS collection and add default admin user (make note of temp password):

sudo python /data/crits/manage.py create_default_collections
sudo python /data/crits/manage.py users  -a -u admin -e admin@localhost.com -R UberAdmin

Verify Access (https://critsserver_ip) and change your temp password:

Done! Will post how to install and enable crits_services in the near future.

2 Likes

Hi Mike! I followed your instructions step by step. I have spent the better part of two days troubleshooting and trying to figure out why I can’t get crits working. Every single step works except for the very last one where you add a user. No matter how I go about it, it always fails and I’m kinda at a loss so I wanted to reach out to you. I can send the error code that I am getting as well if that helps. I reached out to the crits github page and they said the issue was that my mongoengine needed to be rolled back to an earlier version to match that in the requirements.txt file. If you could offer any advice or help I would greatly appreciate it. Thanks in advance for reading through this and making a great guide! I’m 99% of the way to getting it up and running :slight_smile:

Looks like they have made some updates - found your post here: https://www.reddit.com/r/mongodb/comments/c0qqk9/how_to_roll_back_mongoengine/
You should be able to clear your pip cache / remove mongoengine==X.Y.Z
pip list
to verify and then
pip install mongoengine==0.10.6

Hi Mike thank you very much for the advice! I was able to uninstall the new version of mongoengine and roll back to the 0.10.6 version. However, during the pip install I receive this error message.

“django-tastypie-mongoengine 0.4.6+work4.3 has requirement mongoengine>=0.13.0, but you’ll have mongoengine 0.10.6 which is incompatible.”

Will CRITs still work and if not how can I make these two compatible? Thanks!

I may have figured it out but ill let you know

Hi again. Just following up from my last comment. So I rolled back mongoengine and that fixed all install problems. I was able to follow your guide step by step but cannot get a connection on my other machine to the webserver. I made sure there are no firewall or connectivity issues between the two hosts. I go to the IP address of https://#.#.#.# (my crits server’s IP) and do not get a connection.

From what I understand CRITs runs on 8080 by default and Apache2 uses port 80. Could this be of conflict and do you have any insight as to why I cannot connect? I have not expereinced any errors so far and as far as I can tell the install went through perfectly. Thanks!

Have you checked with tcpdump to see if the requests are hitting your external interface and/or curl on the CRITS server to localhost and if so what are the results?