====== Evaluation Metrics ======

This page will have a list of nice-to-have tests for discussing the unobservability of a DS connection.

====== Unobservability Tests of DeltaShaper connections: ======

  - **When compared to a single baseline video:** 
    - How much is a video different from the baseline when carrying payload?
    - What variables of the payload (overlay size, cell cize, framerate, location, etc) affect unobservability the most?
    - Is there a significant difference in the network traffic between multiple runs of the baseline video? How much?
    - Are we able to stream a payload carrying video which looks alike the baseline video (even between the boundaries of a duplicate run of it)?
  - **When compared to similar (same-user) baseline videos:**
    - How much is the difference between similar videos of the same user? Can we grab several videologs and establish a baseline for that user's behaviour? (assuming, let's say, he always places a call on his living room.)
    - If that baseline can be established, would the window for inserting payload be more relaxed? (like, using more space on the frame for placing an overlay or changing cell sizes?)
  - **When compared to a multiple user baseline videos:**
    - How much can we change the payload features assuming that a DS call stays within a "random" regular Skype call boundaries? Probably more than in the previous scenarios.

**Ultimately:**
  - What is the throughput/observability ratio?

**And also:**
  - How does bandwidth limitation and packet dropping/delay impact a regular video traffic?
  - How does bandwidth limitation and packet dropping/delay impact DS unobservability? Does a previously unobservable flow become observable?
===Expectations:===
  - Try an aggressive unobservability combo (meaning less throughput) when dealing with a given call sample (not interesting, a call is usually different);
  - Try a mildly aggressive unobservability combo to suit the baseline of a given user.
  - Try an relaxed unobservability combo (meaning more throughput) to suit a general "Skype call baseline".

{{ expect.jpg }}

====== Resilience Tests of DeltaShaper connections: ======

  - How does bandwidth limitation impact DS covert channel? Does error rate increase with less bandwidth? (Maybe, more quantization on codec)
  - How does packet dropping/delay impact DS covert channel? Is payload recognition affected with packet loss/delay? 
  - If the censor effectively interrupts the video feed, how long does it need to do it to do harm to a DS connection? How much time does DS need to recover from an active attack?