buelfhood / SOCO-Java-CODEBERTA-CONTRASTIVE-PAIRS-E1-B16-LR2e-05-Split0.1

SentenceTransformer based on huggingface/CodeBERTa-small-v1

This is a sentence-transformers model finetuned from huggingface/CodeBERTa-small-v1 . It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: huggingface/CodeBERTa-small-v1
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'RobertaModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("buelfhood/SOCO-Java-CODEBERTA-CONTRASTIVE-PAIRS-E1-B16-LR2e-05-Split0.1")
# Run inference
sentences = [
    '\n\nimport java.io.*;\nimport java.net.*;\nimport java.misc.BASE64Encoder;\n\npublic class Dictionary\n{\n  public Dictionary()\n  {}\n\n  public boolean fetchURL(String urlString,String username,String password)\n  {\n     StringWriter  sw= new StringWriter();\n     PrintWriter  pw = new PrintWriter();\n     try{\n       URL url=new URL(urlString);  \n       String userPwd= username+":"+password;\n\n       \n       \n       \n       \n\n       BASE64Encoder encoder = new BASE64Encoder();\n       String encodedStr = encoder.encode (userPwd.getBytes());\n       System.out.println("Original String = " + userPwd);\n\t System.out.println("Encoded String = " + encodedStr);\n\n       HttpURLConnection huc=(HttpURLConnection) url.openConnection();  \n       huc.setRequestProperty( "Authorization"," "+encodedStr);   \n       InputStream content = (InputStream)huc.getInputStream();\n       BufferedReader in   =\n       new BufferedReader (new InputStreamReader (content));\n       String line;\n       while ((line = in.readLine()) != null) {\n         pw.println (line);\n       System.out.println("");\n       System.out.println(sw.toString());\n       }return true;\n     } catch (MalformedURLException e) {\n       pw.println ("Invalid URL");\n       return false;\n     } catch (IOException e) {\n       pw.println ("Error  URL");\n       return false;\n     }\n\n  }\n\n  public void getPassword()\n  {\n     String dictionary="words";\n     String urlString="http://sec-crack.cs.rmit.edu./SEC/2/";\n     String login="";\n     String pwd=" ";\n\n     try\n     {\n       BufferedReader inputStream=new BufferedReader(new FileReader(dictionary));\n        startTime=System.currentTimeMillis();\n       while (pwd!=null)\n       {\n         pwd=inputStream.readLine();\n         if(this.fetchURL(urlString,login,pwd))\n         {\n            finishTime=System.currentTimeMillis();\n           System.out.println("Finally I gotta it,  password is : "+pwd);\n           System.out.println("The time for cracking password is: "+(finishTime-startTime) + " milliseconds");\n           System.exit(1);\n         } \n\n       }\n       inputStream.close();\n     }\n     catch(FileNotFoundException e)\n     {\n       System.out.println("Dictionary not found.");\n     }\n     catch(IOException e)\n     {\n       System.out.println("Error  dictionary");\n     }\n  }\n\n  public static void main(String[] arguments)\n  {\n     BruteForce bf=new BruteForce();\n     bf.getPassword();\n  } \n}',
    '\n\nimport java.io.*;\nimport java.*;\n\npublic class BruteForce \n{\n   public static void main(String args[]) \n   {\n      String s = null;\n      String basic_url = "http://sec-crack.cs.rmit.edu./SEC/2/";\n\n      \n      String alphabets = new String("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ");\n     \n      String password = null;\n      int len = 0;\n      int num_tries = 0;\n\n      len = alphabets.length();\n      \n      \n      for (int i=0; i<len; i++)\n      {\n         for (int j=0; j<len; j++)\n\t {\n            for (int k=0; k<len; k++)\n\t    {\n               try \n               {\n                  \n                  password = String.valueOf(alphabets.charAt(i)) + String.valueOf(alphabets.charAt(j)) + String.valueOf(alphabets.charAt(k));\n               \n                  System.out.print(alphabets.charAt(i)); \n                  System.out.print(alphabets.charAt(j)); \n                  System.out.println(alphabets.charAt(k));      \n\n                  \n                  Process p = Runtime.getRuntime().exec("wget --http-user= --http-passwd=" + password + " " + basic_url);\n                  \n                  BufferedReader stdInput = new BufferedReader(new \n                     InputStreamReader(p.getInputStream()));\n\n                  BufferedReader stdError = new BufferedReader(new \n                     InputStreamReader(p.getErrorStream()));\n\n                  \n                  while ((s = stdInput.readLine()) != null)\n                  {\n                     System.out.println(s);\n                  }\n                  \n                  \n                  while ((s = stdError.readLine()) != null)\n                  {\n                     System.out.println(s);\n                  }\n                  \n                  try\n\t\t      {\n                     p.waitFor(); \n                  }\n                  catch (InterruptedException g)  \n                  {\n                  }  \n\n                  num_tries++;\n                  \n                  if((p.exitValue()) == 0)\n                  {       \n                     System.out.println("**********PASSWORD IS: " + password);\n\t             System.out.println("**********NUMBER OF TRIES: " + num_tries);\n                     System.exit(1);\n                  }\n               }\n               catch (IOException e)\n               {\n                  System.out.println("exception happened - here\'s what I know: ");\n                  e.printStackTrace();\n                  System.exit(-1);\n               }\n            }\n         }\n      }\n   }\n}\n\n',
    '\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nimport java.io.*;\nimport java.net.*;\nimport java.net.URL;\nimport java.net.URLConnection;\nimport java.util.*;\n\npublic class BruteForce {\n\n   public static void main(String[] args) throws IOException {\n\n      \n      int start , end, total;\n      start = System.currentTimeMillis(); \n\n      String username = "";\n      String password = null;\n      String host = "http://sec-crack.cs.rmit.edu./SEC/2/";\n\n      \n      \n      String letters = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";\n      int lettersLen = letters.length(); \n      int passwordLen=3; \n\n      int passwords=0; \n      int twoChar=0; \n\n      url.misc.BASE64Encoder base = new url.misc.BASE64Encoder();\n      \n\n      \n      String authenticate = ""; \n      String realm = null, domain = null, hostname = null;\n       header = null; \n\n      \n      int responseCode;\n      String responseMsg;\n\n      \n      int temp1=0;\n      int temp2=0;\n      int temp3=0;\n\n\n      \n      \n      \n      for (int a=1; a<=passwordLen; a++) {\n        temp1  = (int) Math.pow(lettersLen, a);\n         passwords += temp1;\n         if (a==2) {\n            twoChar = temp1; \n         }\n      }\n\n      System.out.println("Brute Attack  " + host + " has commenced.");\n      System.out.println("Number of possible password combinations: " + passwords);\n\n\n      int i=1; \n\n       {\n         try {\n            \n            URL url = new URL(host);\n            HttpURLConnection httpConnect = (HttpURLConnection) url.openConnection();\n\n            \n            if(realm != null) {\n\n               \n               if ( i < lettersLen) {\n                  \n\n                   password = letters.substring(i, (i+1));\n\n               } else if (i < (lettersLen + twoChar)) {\n                   \n\n                   \n                   temp1 = i / lettersLen;\n                   password = letters.substring((-1), start );\n\n                   \n                   temp1 = i - ( temp1 * lettersLen);\n                   password = password + letters.substring(temp1, (+1));\n\n               } else {\n                   \n\n                   \n                   temp2 = i / lettersLen;\n                   temp1 = i - (temp2 * lettersLen);\n                   password = letters.substring(temp1, (+1));\n\n                   \n                   temp3 = temp2; \n                   temp2 = temp2 / lettersLen;\n                   temp1  = temp3 - (temp2 * lettersLen);\n                   password = letters.substring(temp1, (+1)) + password;\n\n                   \n                   temp3 = temp2; \n                   temp2 = temp2 / lettersLen;\n                   temp1 = temp3 - (temp2 * lettersLen);\n                   password = letters.substring(temp1, (+1)) + password;\n\n               } \n\n               \n               \n               authenticate = username + ":" + password;\n               authenticate = new String(base.encode(authenticate.getBytes()));\n               httpConnect.addRequestProperty("Authorization", " " + authenticate);\n\n            } \n\n            \n            httpConnect.connect();\n\n            \n            realm = httpConnect.getHeaderField("WWW-Authenticate");\n            if (realm != null) {\n               realm = realm.substring(realm.indexOf(\'"\') + 1);\n               realm = realm.substring(0, realm.indexOf(\'"\'));\n            }\n\n            hostname = url.getHost();\n\n            \n            responseCode = httpConnect.getResponseCode();\n            responseMsg = httpConnect.getResponseMessage();\n\n            \n            \n            \n            \n            \n\n            \n            \n            if (responseCode == 200) {\n               \n               end = System.currentTimeMillis();\n               total = (end - start) / 1000; \n\n               System.out.println ("Sucessfully Connected  " + url);\n               System.out.println("Login Attempts Required : " + (i-1));\n               System.out.println("Time Taken in Seconds : " + total);\n               System.out.println ("Connection Status : " + responseCode + " " + responseMsg);\n               System.out.println ("Username : " + username);\n               System.out.println ("Password : " + password);\n               System.exit( 0 );\n            } else if (responseCode == 401 && realm != null) {\n               \n               \n               \n               if (i > 1) {\n\n               }\n            } else {\n               \n               \n                System.out.println ("What the?... The server replied with  unexpected reponse." );\n               System.out.println (" Unexpected Error Occured While Attempting  Connect  " + url);\n               System.out.println ("Connection Status: " + responseCode + responseMsg);\n               System.out.println ("Unfortunately the password could not  recovered.");\n               System.exit( 0 );\n           }\n\n           i++;\n\n        } catch(MalformedURLException e) {\n           System.out.println("Opps, the URL " + host + " is not valid.");\n           System.out.println("Please check the URL and try again.");\n        } catch(IOException e) {\n           System.out.println(", \'t connect  " + host + ".");\n           System.out.println("Please check the URL and try again.");\n           System.out.println("Other possible causes include website is currently unavailable");\n           System.out.println("  have  internet connection problem.");\n        } \n\n     }  while(realm != null); \n\n\n   }\n}',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.2589, 0.2759],
#         [0.2589, 1.0000, 0.2076],
#         [0.2759, 0.2076, 1.0000]])

Evaluation

Metrics

Binary Classification

• Dataset: binary-classification-evaluator
• Evaluated with BinaryClassificationEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 77,328 training samples
• Columns: text_1 , text_2 , and label
• Approximate statistics based on the first 1000 samples:

  	text_1	text_2	label
  type	string	string	int
  details	min: 51 tokens mean: 467.45 tokens max: 512 tokens	min: 51 tokens mean: 464.45 tokens max: 512 tokens	0: ~50.00% 1: ~50.00%

• Samples:

  text_1	text_2	label
  import java.io. ; import java.net. ; import java.misc.BASE64Encoder; public class Dictionary { public Dictionary() {} public boolean fetchURL(String urlString,String username,String password) { StringWriter sw= new StringWriter(); PrintWriter pw = new PrintWriter(); try{ URL url=new URL(urlString); String userPwd= username+":"+password; BASE64Encoder encoder = new BASE64Encoder(); String encodedStr = encoder.encode (userPwd.getBytes()); System.out.println("Original String = " + userPwd); System.out.println("Encoded String = " + encodedStr); HttpURLConnection huc=(HttpURLConnection) url.openConnection(); huc.setRequestProperty( "Authorization"," "+encodedStr); InputStream content = (InputStream)huc.getInputStream(); BufferedReader in = new BufferedReader (new InputStreamReader (content)); String line; while ((line = in.readLine())...	import java.io. ; import java.net. ; import java.misc.BASE64Encoder; public class BruteForce { public BruteForce() {} public boolean fetchURL(String urlString,String username,String password) { StringWriter = new StringWriter(); PrintWriter pw = new PrintWriter(); try{ URL url=new URL(urlString); String userPwd= username+":"+password; BASE64Encoder encoder = new BASE64Encoder(); String encodedStr = encoder.encode (userPwd.getBytes()); System.out.println("Original String = " + userPwd); System.out.println("Encoded String = " + encodedStr); HttpURLConnection huc=(HttpURLConnection) url.openConnection(); huc.setRequestProperty( "Authorization"," "+encodedStr); InputStream content = (InputStream)huc.getInputStream(); BufferedReader in = new BufferedReader (new InputStreamReader (content)); String line; while ((line = in.readLine()) ...	1
  import java.io. ; import java.net. ; import java.misc.BASE64Encoder; public class Dictionary { public Dictionary() {} public boolean fetchURL(String urlString,String username,String password) { StringWriter sw= new StringWriter(); PrintWriter pw = new PrintWriter(); try{ URL url=new URL(urlString); String userPwd= username+":"+password; BASE64Encoder encoder = new BASE64Encoder(); String encodedStr = encoder.encode (userPwd.getBytes()); System.out.println("Original String = " + userPwd); System.out.println("Encoded String = " + encodedStr); HttpURLConnection huc=(HttpURLConnection) url.openConnection(); huc.setRequestProperty( "Authorization"," "+encodedStr); InputStream content = (InputStream)huc.getInputStream(); BufferedReader in = new BufferedReader (new InputStreamReader (content)); String line; while ((line = in.readLine())...	import java.net. ; import java.io. ; import java.util.*; public class Dictionary{ private static URL location; private static String user; private BufferedReader input; private static BufferedReader dictionary; private int maxLetters = 3; public Dictionary() { Authenticator.setDefault(new MyAuthenticator ()); startTime = System.currentTimeMillis(); boolean passwordMatched = false; while (!passwordMatched) { try { input = new BufferedReader(new InputStreamReader(location.openStream())); String line = input.readLine(); while (line != null) { System.out.println(line); line = input.readLine(); } input.close(); passwordMatched = true; } catch (ProtocolException e) { } catch (ConnectException e) { System.out.println("Failed connect"); } catch (IOException e) ...	0
  import java.io.InputStream; import java.util.Properties; import javax.naming.Context; import javax.naming.InitialContext; import javax.rmi.PortableRemoteObject; import javax.sql.DataSource; public class WatchdogPropertyHelper { private static Properties testProps; public WatchdogPropertyHelper() { } public static String getProperty(String pKey){ try{ initProps(); } catch(Exception e){ System.err.println("Error init'ing the watchddog Props"); e.printStackTrace(); } return testProps.getProperty(pKey); } private static void initProps() throws Exception{ if(testProps == null){ testProps = new Properties(); InputStream fis = WatchdogPropertyHelper.class.getResourceAsStream("/watchdog.properties"); testProps.load(fis); } } }	import java.io.InputStream; import java.util.Properties; import javax.naming.Context; import javax.naming.InitialContext; import javax.rmi.PortableRemoteObject; import javax.sql.DataSource; public class BruteForcePropertyHelper { private static Properties bruteForceProps; public BruteForcePropertyHelper() { } public static String getProperty(String pKey){ try{ initProps(); } catch(Exception e){ System.err.println("Error init'ing the burteforce Props"); e.printStackTrace(); } return bruteForceProps.getProperty(pKey); } private static void initProps() throws Exception{ if(bruteForceProps == null){ bruteForceProps = new Properties(); InputStream fis = BruteForcePropertyHelper.class.getResourceAsStream("/bruteforce.properties"); bruteForceProps.load(fis); } } }	1

• Loss: ContrastiveLoss with these parameters:

  {
      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
      "margin": 0.5,
      "size_average": true
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 8,592 evaluation samples
• Columns: text_1 , text_2 , and label
• Approximate statistics based on the first 1000 samples:

  	text_1	text_2	label
  type	string	string	int
  details	min: 51 tokens mean: 469.8 tokens max: 512 tokens	min: 51 tokens mean: 461.39 tokens max: 512 tokens	0: ~50.00% 1: ~50.00%

• Samples:

  text_1	text_2	label
  import java.util. ; import java.io. ; public class WatchDog { public static void main(String args[]) { Runtime rt1 = Runtime.getRuntime(); Process prss1= null; try { prss1 = rt1.exec("wget -R mpg,mpeg, --output-document=first.html http://www.cs.rmit.edu./students/ "); }catch(java.io.IOException e){} MyWatchDogTimer w = new MyWatchDogTimer(); Timer time = new Timer(); time.schedule(w,864000000,864000000); } }	import java.util. ; import java.io. ; public class MyTimer { public static void main(String args[]) { Watchdog watch = new Watchdog(); Timer time = new Timer(); time.schedule(watch,864000000,864000000); } }	1
  import java.util. ; import java.io. ; public class WatchDog { public static void main(String args[]) { Runtime rt1 = Runtime.getRuntime(); Process prss1= null; try { prss1 = rt1.exec("wget -R mpg,mpeg, --output-document=first.html http://www.cs.rmit.edu./students/ "); }catch(java.io.IOException e){} MyWatchDogTimer w = new MyWatchDogTimer(); Timer time = new Timer(); time.schedule(w,864000000,864000000); } }	import java.net. ; import java.io. ; import java.util.Vector; import java.util.Date; import java.security. ; public class Dictionary { public static BufferedReader in; public static void main(String[] args) throws Exception { String baseURL = " http://sec-crack.cs.rmit.edu./SEC/2/index.php "; int count=0; Date date = new Date(); startTime=date.getTime(); int LIMITINMINUTES=45; int TIMELIMIT=LIMITINMINUTES 1000*60; boolean timedOut=false; boolean found=false; Vector dictionary=new Vector(readWords()); System.out.println("Words in dictionary: "+dictionary.size()); while (found==false && timedOut==false && dictionary.elementAt(count)!=null) { Date endDate = new Date(); endTime=endDate.getTime(); if (endTime>(TIMELIMIT+startTime)){ System.out.println("Timed out"); timedOut=true; } String password = ""; ...	0
  import java.io.InputStream; import java.util.Properties; import javax.naming.Context; import javax.naming.InitialContext; import javax.rmi.PortableRemoteObject; import javax.sql.DataSource; public class MailsendPropertyHelper { private static Properties testProps; public MailsendPropertyHelper() { } public static String getProperty(String pKey){ try{ initProps(); } catch(Exception e){ System.err.println("Error init'ing the watchddog Props"); e.printStackTrace(); } return testProps.getProperty(pKey); } private static void initProps() throws Exception{ if(testProps == null){ testProps = new Properties(); InputStream fis = MailsendPropertyHelper.class.getResourceAsStream("/mailsend.properties"); testProps.load(fis); } } }	import java.io.InputStream; import java.util.Properties; import javax.naming.Context; import javax.naming.InitialContext; import javax.rmi.PortableRemoteObject; import javax.sql.DataSource; public class BruteForcePropertyHelper { private static Properties bruteForceProps; public BruteForcePropertyHelper() { } public static String getProperty(String pKey){ try{ initProps(); } catch(Exception e){ System.err.println("Error init'ing the burteforce Props"); e.printStackTrace(); } return bruteForceProps.getProperty(pKey); } private static void initProps() throws Exception{ if(bruteForceProps == null){ bruteForceProps = new Properties(); InputStream fis = BruteForcePropertyHelper.class.getResourceAsStream("/bruteforce.properties"); bruteForceProps.load(fis); } } }	1

• Loss: ContrastiveLoss with these parameters:

  {
      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
      "margin": 0.5,
      "size_average": true
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy : steps
• per_device_train_batch_size : 16
• per_device_eval_batch_size : 16
• learning_rate : 2e-05
• num_train_epochs : 1
• warmup_ratio : 0.1
• fp16 : True
• batch_sampler : no_duplicates

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.11.11
• Sentence Transformers: 5.1.1
• Transformers: 4.56.2
• PyTorch: 2.8.0.dev20250319+cu128
• Accelerate: 1.10.1
• Datasets: 4.1.1
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

ContrastiveLoss

@inproceedings{hadsell2006dimensionality,
    author={Hadsell, R. and Chopra, S. and LeCun, Y.},
    booktitle={2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)},
    title={Dimensionality Reduction by Learning an Invariant Mapping},
    year={2006},
    volume={2},
    number={},
    pages={1735-1742},
    doi={10.1109/CVPR.2006.100}
}