How to create a new Obyte Bot project from existing local project


1. Copy existing project into a new directory.

Copy existing project directory usingWindows Explorer. Rename new directory with the name of your project.

2. Remove .git directory & package-lock.json files

Remove .git directory from the root directory of the new project.

Remove pacakge-lock.json files from the client and server project directories.

3. Replace references to the old project

4.1. Server

Update package.json and conf.js files in the server directory. Replacing references to the old project.

Check .env file in the server, to make sure env is set to test.

4.1. Client

Update README.md, quasar.conf.js and package.json files in the client directory. Replacing references to the old project.

At this stage, you may also want to change Application Name in the client/src/layouts/MainLayout.vue file.

4. Install libraries

4.1. Server

At Command Prompt navigate to the new project's server directory and run npm install to install required libraries. This will also create SQLight database.

Test by running node server.js

4.2. Client

At Command Prompt navigate to the new project's client directory and run npm install to install required libraries.

You will see some warnings, run npm audit fix command to fix it.

Test by running quasar dev and viewing in the browser using http://localhost:8080/#/home

5. Setup new Firebase project and users

5.1. Create a new Firebase Project

In your browser go to firebase.google.com and click Add Project - this will start the Project set-up wizard. Provide your project name (e.g. My New App). Disable Google Analytics option and click on Create Project button.

5.2. Update Firebase client config of your project

Click on the Add Firebase to your app web icon . Specify app's nickname and click Register app button. Copy firebaseConfig object and use it to update client/src/boot/firebase.js file.

5.3. Setup Firebase Authentication and add Firebase Users

Click on Authentication option from the Firebase menu. Then click on Set up sign-in method and select email/password authentication method and Enable it.

Click on Users option and add some users for testing.

5.4. Generate Private Key for Server Side Firebase Authentication set-up

To authenticate a service account and authorize it to access Firebase services, you must generate a private key file in JSON format.

In the Firebase console, click on config/setting icon next to the Project Overview Select Project settings menu option

Go to the Service accounts tab and click Generate New Private Key , then confirm by clicking Generate Key button.

Securely store the JSON file containing the key.

Create a new env variable pointing to the location of the key. Update server/services/firebase-service.js file with the name of the new variable.

You will also be provided with the Admin SDK config snippet. And Firebase service account - copy it and use it to update databaseURL in the server/services/firebase-service.js file.

5.5. Test

Stop server, close and re-open command propmt window to allow for new Env Variable to be recognised. Start server and test Firebase authentication by logging in as one of the users.

Follow instructions in the Full Stack App tutorial.

6. Create custom db table(s)

Update server/db.sql file, renaming db table(s).

Also update server/routes/pairing.js, server/bot/attest_utils.js and server/bot/bot_utils.js files with a new table name.

Run node db_import.js at the command propmpt in your server directory. This should create your custom table(s).

Test by checking the database using the SQLite Studio application. Open it and choose Add Database menu option. Add your new project database. And you should see your new tables.

7. Install obyte js & make other changes

If you want to use obyte js librarty, e.g. to search attestation data on the DAG install it by running npm i obyte --save at the server directory command prompt.

Make necessary changes & test.

8. Create GitHub Repo and commit

Login to GitHub and create a new repository.

Copy the URL.

At command prompt, go to your local project directory and create an empty git repository and connect it to the github:

git init
git remote add origin  REMOTE_URL

Than, make your 1st commit:

git add .
git commit -m "1st commit"
git push origin master

If you get fatal: not a git repository (or any of the parent directories): .git error, type git init to create an empty git repository in your local directory.


9. Deploy to the Digital Ocean

9.1. Digital Ocean - server setup

See https://www.youtube.com/watch?v=RE2PLyFqCzE for a complete tutorial.

1. Download Putty & PuttyGen

Download Putty and PuttyGen onto your Windows machine if you do not have them already.

2. Generate SSH keys

Open PuttyGen and click on the Generate button.

Click on the Save public key button and save the file locally, e.g. DOFSApublickey1.txt

Then enter a passphrase (optional) and click on the Save private key button and save the file locally, e.g. DOFSAprivatekey1.ppk

Keep PuttyGen open.

3. Sign up with Digital Ocean

Open Digital Ocean account.

4. Setup a DO Droplet

Login into your new DO account.

Create a project, eg. My Full Stack App

Create Droplet for your project as following:

  • Image: Ubuntu
  • Size: $5/mo
  • Choose a datacenter closest to you
  • Authentication: click on the New SSH Key button. Go to the PuttyGen Window and copy the entire content of the Public Key text box. Paste it into the Add Public SSH key window on the Digital Ocean page. Give it a name: e.g. FSApublicKey1 and click on the Add SSH Key button.
  • Host name: e.g. myFullStackApp

Click Create. This will set-up your server. Make a note of the IP Address. E.g. 167.172.59.209

Click on the droplet name to get to the DO panel

5. Connect to server via Putty

Open Putty.

Add IP address of your new server.

Create connection profile for your new server on Putty:

Click on Connection --> Data menu option and enter root in the Auto-login user name

Click on SSH --> Auth menu options and upload your private key file DOFSApublickey1.ppk

Go back to the Session tab and enter IP Address 167.172.59.209 of your new Server. Then enter the name of your session, e.g. My Full Stack App at DIGITAL OCEAN and save it by clicking on the Save button.

Click Open. You will see a warning message, click Yes. Enter your passphrase. You are now connected to your new server.

9.2. Digital Ocean - setting up new user and disabling root access
1. Create a new user

Using Putty connect to your server and type:

adduser myuser

Give user admin preveliges

usermod -aG sudo myuser

Test that the user is assign to sudo group:

id myuser

Sign in as your new user:

sudo su -myuser

Check by typing:

whoami

2. Authorise SSH key for the new user

Create a .ssh directory under home directory:

cd ~
mkdir ~/.ssh
chmod 700 ~/.ssh

Save public SSH key for your new user.

Create authorized_keys file:

nano ~/.ssh/authorized_keys

Copy your public key into it. Note, that if you are copying your key from your public key .txt file , type 'ssh-rsa ' (including space) followed by the key from the .txt public key file. Ignore first 2 lines and the last line of comments in the public key .txt file. Make sure that the key is not split across multiple lines, but is one line.

Change permissions on this file:

chmod 600 ~/.ssh/authorized_keys

Restart SSH service:

sudo service ssh restart

Exit:

exit

Close Putty connection.

3. Configure Putty connect for new user

Open Putty. Select your connection and click Load.

Then go to Connection --> Data and change Auto-login username from root to myuser.Go back to the Session. Click Save followed by Open. You should be able to connect by entering the passphrase.

4. Disable Root and Password Login

Your password login should be disabled based on your droplet creation options.

To check, login as your new user using Putty and open sshd_config file:

sudo nano /etc/ssh/sshd_config

Use ctrl+w to search for the following:

PermitRootLogin
PasswordAuthentication

And set both options to no.

Reload sshd with this command:

sudo systemctl reload sshd

Test by going to Putty and pasting the IP address and clicking Open. You should not be able to connect as a root or as your user.

Test by changing user to root on your saved connection. You still should not be able to connect.

Change the user back to your user and Save.

9.3. Digital Ocean - install Node.js, npm, Git, connect to GitHub using SSH Key
1. Install Node.js and npm

Using Putty connect to your server and type following commands to install latest version of node js and npm:

cd ~
curl -sL https://deb.nodesource.com/setup_10.x -o nodesource_setup.sh
sudo bash nodesource_setup.sh
sudo apt install nodejs
sudo apt install build-essential

Check versions of node and nvm:

node -v
npm -v

For more information go to the https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-ubuntu-18-04 for a complete tutorial.

2. Install git

Install git with the following command:

sudo apt-get install git

Check git version:

git --version

9.4. Digital Ocean - create SSH Key and connect to GitHub using it
1. Create SSH Key for the Github

Using Putty connect to your server and type following commands to generate SSH Key for the Github:

cd ~
ssh-keygen -t rsa -C "natalie.seltzer@gmail.com"

2. Copy SSH Public Key

Copy the path to the newly created ssh public key and use it to display it by typing the following command on the server:

cat /home/myuser/.ssh/id_rsa.pub

This will print the new ssh public key. Highlight and copy the key.

Note, that the key starts with ssh-rsa followed by space, followed by the key, followed by space, followed by your email address. Copy the key.'

See Full Stack App Tutorial for alternative using WinSCP.

3. Add SSH Public Key to your GitHub repository

Login to your Github and go to your project repository. Then go to Settings and click on the Deploy Key left hand menu option. Then click on the Add deploy key button add new read-only SSH Key (if the key you have copied has your email address at the end - remove it).

Alternatively, login to your Github and go to your project repository. Then click on Copy button and select SSH option. Click on the link to add SSH Key left . Add key including ssh-rsa, followed by space, by key, by another space and your email address.

4. Test connection to the GitHub from the server

To test connection to your GitHub type the following command on your server:

ssh -vT git@github.com

9.5. Digital Ocean - deploy code from GitHub, install Lib, SQLite
1. Clone your Github project repository onto the remote server

First, make sure that your GitHub Repo is up to date. Then use Putty to connect to your remote server.

Clone your project repository using SSH key:

cd ~
git clone git@github.com:CriptoGirl/FullStackApp.git

You will be asked for a passphrase.

For more info and alternatives see Full Stack App Tutorial.


2. Install server & client

Install server & client dependencies:

cd FullStackApp
cd server
npm install
cd ..
cd client
npm install

3. Deploying SQLite

On a server go to to your project's /server directory and run the following command:

node db_import.js

This will create SQLite db and run your script to create custom tables (if any).

9.6. Digital Ocean - Firebase config
1. WinSCP

If you do not have WinSCP already installed locally, download it, accepting the defaults and saying yes to import connections from Putty.

Open WinSCP and create a new connection. Enter server IP address as a host name and upload the private key file by going to the SSH / Athenticate section of the Advance settings. Login.

2. Create Config dir on the Digital Ocean Server

Use Putty to connect to your remote server, then create Config directory under your user directory:

cd ~
mkdir Config

3. Copy Firebase Admin SDK to the Digital Ocean Server

Use WinSCP to copy C:\Users\natal\node js\FirebaseAuth\my-full-stack-app-firebase-adminsdk-3l3z4-2b81d51460.json to FSA_Firebase_config.json file in the ~/Config directory.

4. Optional: set up temp Env Variables and Test

Set up temp env variable to store the location of the Firebase config file. Check the name of the variable used by your application as referred to in the /server/services/firebase-service.js file. Execute the following commands to set temp env variable that will help with testing, but will be gone when you close your server connection. We will set this up to last later using pm2.

cd ~
export GOOGLE_APPLICATION_CREDENTIALS="/home/myuser/Config/FSA_Firebase_config.json"
echo $GOOGLE_APPLICATION_CREDENTIALS

Test by going to the /server directory and starting the server:

node server

9.7. Digital Ocean - Add Server Location to Quasar Client Config
1. Add Digital Ocean IP to Quasar Process Env variable

Open /client/quasar.conf.js file and go to build: section and add remote server location Process Env Variable, e.g.

build: {
  env: ctx.dev
    ? { // so on dev we'll have
      SERVER_URL: JSON.stringify(`http://localhost:8081/`)
    }
    : { // and on build (production)
      SERVER_URL: JSON.stringify(`http://167.172.59.209:8081/`)
    },

2. Point Axios to use Quasar Process Env variable

Make sure that client/src/services/api.js file, uses SERVER_URLENV Variable, as following:

baseURL: process.env.SERVER_URL,

Test that your app still works in dev mode locally.

quasar dev

3. Build Quasar SPA

Build SPA by going to client directory and typing:

quasar build

This creates a new dist/spa folder under our client folder.

4. Post latest code to Github

Update /client/.gitignore file and remove/comment out /dist directory line.

Then, go to the application directory:

git add .
git commit -m "client build for remote deployment"
git push -u origin master

5. Deploy latest code to Digital Ocean

Using PuTTY connect to the server, go to your project dir and run following commands:

git pull origin master

9.8. Digital Ocean - setting up PM2 Process Manager
1. Install PM2 process manager

We are going to use PM2 to run our application as a service in a background. Install it on your server by running:

cd ~
sudo npm install -g pm2

2. Configure PM2 using ecosystem file

To generate a sample ecosystem.config.js file in your user directory type:

cd ~
pm2 ecosystem

Amend it using nano editor:

cd ~
nano ecosystem.config.js

Update ecosystem.config.js file to look like following:

module.exports = {
  apps : [{
    name: 'FSA',
    script: '/home/myuser/fsa/server/server.js',
    autorestart: true,
    watch: true,
    max_memory_restart: '1G',
    env: {
      NODE_ENV: 'development'
    },
    env_test: {
      NODE_ENV: 'test',
      FIREBASE_ADMIN: '/home/myuser/Config/FSA_Firebase_config.json'
    },
    env_production: {
      NODE_ENV: 'production'
    }
  }]
};

Changing name, script, FIREBASE env variable name and location.

3. Start the server with PM2

Start the server as a background process in test mode with pm2 run:

pm2 start ecosystem.config.js --env test

4. Check that the Server is running sucessfully

Test that the server is running and check for errors:

pm2 list
pm2 logs --lines 200
pm2 stop FSA

To stop the server:

pm2 stop FSA

For more information on click here. Click here for a list of pm2 commands.

5. Resolving Issues

If you get the following error:

Error: Looks like you are loading multiple copies of ocore, which is not supported.

Check if there is a node_modules directory under server/node_modules/headless-obyte directory. cd to the server/node_modules/headless-obyte directory and remove node_modules directory and its sub-directories:

rm -r node_modules

9.9. Digital Ocean - Install & configure Ngnex
1. Install Nginx

Install Nginx on the server:

cd ~
sudo apt-get install nginx

Check that Nginx is running:

service nginx status

Use Cntrl-C to exit from the status window.

Test by going to your browser and typing your server's IP Address, e.g.

167.172.59.209

You should see:

Welcome to nginx!

2. Configure Nginx

Add path to your SPA client index.html file to the Nginx config:

nano /etc/nginx/sites-available/default

Amending root path as following:

root /home/myuser/myfullstackapp/client/dist/spa;

Replacing pass with the pass to your project spa directory.

If you do not have edit right to this file change file permissions by navigating to the /etc/nginx/sites-available directory and changing permissions as following:

cd /etc/nginx/sites-available
ls -la
sudo chmod 777 default
ls -la

Then edit the file and save.

Change permissions back with the following command:

sudo chmod 644 default
ls -la

You should NOT need to restart Nginx but check that it is running as described above.

If you need to restart Nginx you can do it with the following command:

sudo service nginx restart

To check status of Nginx type:

sudo service nginx status

Type q to exit.

3. Test in the Browser

Use IP addresss of your server to test in the Browser.

9.10. Digital Ocean - UFW Firewall config
1. Instaling UFW Firewall

UFW firewall is installed by default on Ubuntu. To check if it is install, type:

sudo ufw status verbose

If it has been uninstalled for some reason, you can install it with:

sudo apt install ufw

For more info click here.

2. Setting up Default Policies

The first rules to define are your default policies. These rules control how to handle traffic that does not explicitly match any other rules. By default, UFW is set to deny all incoming connections and allow all outgoing connections. You also want to allow incoming SSH connections so you can connect to and manage your server. To set the defaults used by UFW, use these commands:

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh

Please note, that it is very important to allow ssh connection, without it you would not be able to connect to your server.

3. Enabling UFW

To enable UFW, use this command:

sudo ufw enable

You will receive a warning that says the command may disrupt existing SSH connections. Respond to the prompt with y and hit ENTER.

The firewall is now active. Run the following command to see the rules that are set:

sudo ufw status verbose

4. Adding ports

Test that you can nor longer access your application from the browser.

Allow other ports used by your application:

cd ~
sudo ufw allow 80
sudo ufw allow 8081
sudo ufw allow 443

5. Turn on Firewall Logging and Test

Turn on firewall logging:

sudo ufw logging on

Test by running your application in a browser.

To see firewall logs:

sudo tail -f /var/log/ufw.log

6. To disable Firewall:

If at any point you need to disable the firewall:

sudo ufw disable
sudo ufw status

6. Optional: using IPv6 with UFW

10. Digital Ocean - Re-deploy latest code

1. Push new version to GitHub

If you made any changes to the client front end code, remember to build a new SPA by executing the following command in your local /client directory:

quasar build

Then go to your local project directory and deploy to GitHub:

git add .
git commit -m "some changes"
git push origin master

2. Deploy to the Digital Ocean

Use Putty to connect to your remote server, then stop the server:

cd ~
pm2 stop FSA
pm2 list

G go to your project directory and type following commands:

git pull origin master

If you have made any changes to your remote files and do not want to keep them, you can reset server git by typing following command in the project folder:

git reset --hard
git pull origin master

3. Installing Libraries

If you have added new libraries to the client, to to the project directory and:

cd client
npm install

If you have added new libraries to the server, to to the project directory and:

cd server
npm install

4. Starting the Server

Go to the home directory and restart the server:

cd ~
pm2 start ecosystem.config.js --env test
pm2 list
pm2 logs --lines 200

5. Resolving Issues

If after installing new librarites in the server directory you get the following error:

Error: Looks like you are loading multiple copies of ocore, which is not supported.

Check if there is a node_modules directory under server/node_modules/headless-obyte directory. cd to the server/node_modules/headless-obyte directory and remove node_modules directory and its sub-directories:

rm -r node_modules

11. Other Useful Commands

1. Other useful Git commands:

If you realised that you have added wrong file (e.g. config or notes) to your local git staging area with the git add . command. You can clear (unstage) with the following command:

git reset

To check what is in your git staging area, run the following command:

git status

If you have made any changes to your remote Server files and do not want to keep them, you can reset server git by typing following command in the project folder:

git reset --hard
git pull origin master